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SAN FRANCISCO— {BW HealthWire) --March 17, 1997~UC San Francisco AIDS 
specialists announced today (March 17) that anyone with access to a 
computer and to the "world wide web" can now obtain the most comprehensive, 
credible and trustworthy HIV/AIDS information available online with just a 
click of the mouse. 

The top-rated UCSF AIDS Program at San Francisco General Hospital and 
the UCSF Center for AIDS Prevention Studies (CAPS), programs of the UCSF 
AIDS Research Institute, have created "HIV InSite" — a one-stop shop for 
reliable, peer-reviewed AIDS information and the only website in existence 
that contains research written, edited and maintained by frontline AIDS 
researchers from a health sciences institution. 

The address for the site is http://hivinsite.ucsf.edu . 

"Reliable, timely information is essential to developing effective 
responses to the HIV epidemic, " said Paul Volberding, MD, UCSF professor of 
medicine and director of the UCSF AIDS Program at SFGH. "The internet 
offers the potential to arm decision makers throughout the world with 
accurate up-to-date information, fostering the development of sound 
policies, enhancing prevention program development, and providing people 
living with HIV with information about treatments that may extend their 
lives. " 

Unfortunately, AIDS information reported in the popular media is 
often incomplete or inaccurate, and those most in need of information may 
sometimes become overwhelmed or lost in cyberspace, according to Thomas 
Coates, PhD, director of CAPS and the UCSF AIDS Research Institute. 

"Information found at HIV Insite is reviewed and presented by the top 
thinkers in the AIDS field," said Coates. "No other institution can offer 
such depth in this area." 

"With the launch of HIV InSite, patients, physicians, researchers, 
policymakers, health service providers, community organizers and 
journalists can be ensured that they are accessing the most reliable and 
current online resource for AIDS knowledge," Coates added. 

Surfers of internet and users of "HIV InSite" will benefit from the 
breadth of in-depth information on the site — from treatment, clinical 
drug trials, epidemiology and basic research to social and policy issues, 
prevention programs, population subgroups, and ethics. A distinct advantage 
of HIV InSite is the ability of the user to "see the big picture" of the 
AIDS topic he or she is researching, thereby adding value to the search, 
said Nicole Mandel, project manager of the web site. 

For example, a reporter in New Mexico who might be writing a story 
about HIV infection among teenage runaways may dialup the internet to find 
out prevalence statistics. Accessing HIV InSite, the reporter will not only 
find out HIV rates among this group, which can be broken down by regions or 
states, but will find facts on homelessness and AIDS, a research study on 
HIV prevention for high-risk youth, and lists of local and national 
advocacy groups working with young people. 

Another scenario might include an outreach worker at a 
community-based organization in North Dakota who is working with prisoners 
for the first time. Visiting HIV InSite, he or she will find a list of 
prison-based HIV prevention projects, an evaluation of a post-incarceration 
followup programs at San Quentin, and a review of prison condom 
distribution programs. 

"The comprehensive nature of HIV Insite adds value to the 
researchers' information search," Mandel said. 

Pages or subcategories of HIV Insite include: 
Medical 



This area provides comprehensive, practical and state-of-the-art 
information to meet the needs of health care providers, * clinical 
researchers, health care policymakers and those living with HIV disease . 
Highlights include a "trials search" — a user-friendly database of all HIV 
clinical trials in the U.S. in which users may search by medical 
condition , drug or treatment name, or location to find locations of 
appropriate clinical trials. Another feature is the AIDS Knowledge Base 
— considered by many researchers to be the "bible" of HIV disease — a 
1600-page online textbook that is edited by AIDS specialists from UCSF and 
SFGH. In addition, users may explore case studies, treatment guidelines, 
and clinical fact sheets. 
Prevention 

The goal of this section is to help service providers, researchers, 
educators and others build stronger programs and studies in the effort to 
prevent HIV infections. HIV InSite offers detailed resources to answer some 
of the most pressing needs in prevention, including pages on epidemiology, 
prevention basics, frequently asked questions and their answers, resources 
and links to basic information about HIV/AIDS and safer sex, and HIV 
prevention interventions that have been evaluated for effectiveness and 
have been published in scientific journals and government reports. 
Social Issues 

This section is a searchable, comprehensive index of HIV social and 
ethical issues, resources and analysis. The site contains policy resource 
materials, such as statements, guidelines, reports and analysis, and new 
developments on topics ranging from adolescents to workplace issues. The 
section catalogues and organizes policy information and resources from 
numerous sources, including federal, state and local legislatures and 
agencies, the courts, professional associations, health policy research 
organizations, AIDS community-based organizations and advocacy groups. 
U.S. Map 

A map of the United States includes state-by-state statistics of the 
epidemic, as well as links to AIDS agencies and resources available in 
individual states. The AIDS epidemic in the U.S. varies by metropolitan 
area and state, and accounts for a total of 7% of all cases of HIV 
infection in the world. 

HIV InSite will expand this geographic guide to HIV to include other 
areas around the globe, but in the meantime has links to other HIV InSite 
pages or other websites that have international data. HIV-infected 
individuals will also be able to locate clinical drug trials in their 
geographic area by keying in their medical information either through the 
U.S. map or through the clinical section of HIV InSite. 
Key Topics 

This section includes comprehensive information on key HIV and AIDS 
topic areas. Users may look here for news, articles, opinion pieces, 
documents, abstracts, bibliographies, contacts and other information. Among 
topics included in the exhaustive list are: advances in treatment, African 
Americans, condoms, discrimination, health care workers, international, 
lesbians. Native American, needle exchange, Ryan White Care Act, schools, 
sex workers, tuberculosis, vaccines and women. 

The UCSF AIDS Program at San Francisco General Hospital, founded in 
1983, is one of the oldest and largest programs of its kind in the world. 
With an internationally-renowned faculty and extensive clinical and 
research activities, the AIDS program is on the frontline of advances in 
clinical care for people with HIV. The program received the number one 
ranking from U.S. News and World Report in 1996 for the sixth year in a 
row. 

CAPS, established in 1986, is the largest consolidated research 
effort in the country focusing on the social and behavioral aspects of AIDS 
prevention and early intervention. Research activities are carried out 
locally, nationally and internationally. 

CAPS also conducts ethical studies and policy analyses of 
AIDS-related issues, provides a program of technology transfer and exchange 
with community-based organizations, and trains U.S. and international 
scientists in AIDS prevention. CAPS is made possible through grant funding 
from the Office of AIDS and the National Institute of Mental Health. 

HIV InSite is made possible by a grant from the Henry J. Kaiser 
Family Foundation, an independent health care philanthropy based in Menlo 
Park, California. The Foundation is not associated with Kaiser Permanente 
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ABSTRACT: In the first part of a two-part article, the author discussed 
the problem faced by phys icians and other health care workers trying to 
keep up with the explosion of new medical ' information , In this, the second 
part, he discusses in detail the contribution that computers are making to 
the management and dissemination of medical information. Medical knowledge 
stored in computers may be used in four ways, but the author indicates that 
these four ways are interrelated and tend to overlap. One application of 
computer medical knowledge is hypertext. In this system, the reader is not 
limited to reviewing text and illustrations in a particular order, but can 
jump from tidbit to tidbit in any order which suits his need. A drawback of 
such systems is that the freedom of movement within a body of medical 
knowledge may result in the reader becoming lost or perhaps not 
appreciating the relationships between concepts which would be presented 
sequentially in a bound volume. A second, and very important, application 
of computer technology to medical knowledge is simply the maintenance of 
large knowledge files. Though such files might' be called "databases", they 
are in fact knowledge bases and are more akin to reference books than to 
databases. A third contribution of computer technology to medical knowledge 
is specialized teaching aids. These systems are designed not for the 
professional seeking to add to his knowledge or perhaps look up a fact of 
which he is not certain, but rather for people to learn new things. The 
computer provides a presentation which may be individually tailored to the 
needs and abilities of the learner. The last category of computer 
applications to medical knowledge are the decision aids. These systems, 
somewhat akin to expert systems, help the physician by presenting the 
step-by-step sequences of decisions which must be made in the proper 
diagnosis of some symptoms or in the management of continuing treatment of 
patients. (Consumer Summary produced by Reliance Medical Information, Inc.) 

TEXT: 

In the first article I discussed medical knowledge and 
decision-making, the sources that doctors currently use to find this 
knowledge, and some of the drawbacks of these sources. I also described 
computer-based methods for searching bibliographic files. In this paper I 
concentrate on novel computer systems in which medical knowledge is not 
only stored and manipulated as text meaningful to human beings but is also 
encoded as symbols meaningful to computers . 

There are four reasons to encode the knowledge held in a computer 
system: 

* To allow a computer user to move freely between passages of text, 
illustrations, or even sound recordings in a non-linear "hypertext" 
document . 

* To help a computer user find specific details in a large mass of 
medical facts stored in a "knowledge base" , the electronic equivalent of a 
reference book. 

* To allow the computer to teach the user, by itself finding relevant 
details in its knowledge base that the user does not know (a teaching aid) . 

* To allow the computer to help the user solve clinical problems, by 
itself finding details in the knowledge base that are relevant to the 
patient (a medical decision aid) . 

The key properties of hypertext, knowledge bases, and teaching and 
decision aids are listed in the table. These four kinds of system tend to 
overlap: in general, the more helpful a system is in decision-making about 
individual patients, the narrower its coverage of medicine (fig 1) . For 
simplicity, I shall look at each kind in isolation. 

Hypertext and hypermedia 



When we use a textbook, we do not usually start at the beginning and 
read on the end. Instead, we turn to figures or references on other pages, 
refer back to explanations in preceding chapters, and use glossaries and 
appendices to enhance our understanding. When text is stored on computer in 
the conventional way we are denied this instant access that the paper 
version gives us. However, these links can be recovered, and even enhanced, 
by the use of hypertext techniques [1] where links or short-cuts are 
provided between parts of the text. For example, a "definition" link may 
connect a word in the text with its glossary entry, while "see also" links 
draw attention to related sections of text. Frequently these links are 
shown as "buttons" or bold text on the computer screen, and users navigate 
through the system by selecting these items with a pointing device (mouse) . 
Development of these systems can be difficult, especially if there is more 
than one author, since one has to keep track of multiple passages of text 
and their links as the work goes through successivie revisions. Hypertext 
systems are now widely available outside medicine, guiding the uninitiated 
through software packages and tourists through airports. The Gosta's book 
project [2] is a medical hypertext system with 3000 links to help primary 
care physicians browse through the clinical and laboratory features of 
diseases , 

Hypertext alone is no substitute for books in which diagrams or 
photographs are important. Thus, developers are now incorporating 
high-resolution computer graphics, sound, and video pictures to produce 
"hypermedia" systems. These are most widely used in teaching, and I shall 
discuss them in the section on teaching aids. 

The similarity of one "page" to another in computer systems, the lack 
of a real beginning and end, and the fact that some hypertext systems offer 
ten or more links per page can lead to users getting "lost in hyperspace". 
To present a coherent overview and to avoid disorientation, some hypertext 
systems include a "guided tour" or maintain a list of all pages the user 
has viewed. Other systems provide an "overview" showing the structure of 
the whole document. [3] 

Medical knowledge bases 

Databases, the computer equivalent of card indexes, provide an 
efficient means for storing and retrieving large amounts of well structured 
data. If the store contains medical information such as drug dosages or 
disease incidences rather than patient data, the term "medical knowledge 

base " is preferred to database. There is a risk of overlap in terminology 
here, since disease incidences are calculated from a patient database, 
but few systems hold both kinds of information. Most knowledge bases 
include a search program to enable clinicians with modest computer 
experience to identify valid search items from lists and to combine them 
logically to identify relevant material. Medical knowledge bases can be 
stored on a central computer, with remote access by telephone or network 
links, or can be duplicated on floppy disk or compact disk read-only memory 

(CD-ROM) and installed on personal computers. 
[TABULAR DATA OMITTED] 

The difficulties of establishing and maintaining medical knowledge 
bases are identiacal to those associated with paper-based knowledge 
collections, discussed earlier. They include the difficulty of ensuring 
that the details contained are current and correct, that the coverage is 
adequate for clinical use, and that the index uses clinically relevant 
terms. However, two problems that are specific to electronic media are the 
use of symbols rather than a subtle natural language and the need to 
express knowledge in a form that can be read by any program running on any 
computer . 

The London Dysmorphology Database is an example of a medical 
knowledge base distributed on floppy disk. [4] This details the mode of 
inheritance and clinical signs of 2000 congenital malformation syndromes. 
An editorial panel regularly reviews 1000 journals and issues annual 
updates. If congenital malformation interest you, it is difficult to see 
how this system can be bettered. 

Floppy disks are not the ideal vehicle for knowledge bases because 
they are fragile and have limited capacity. A reasonable limit is 6, which 
will contain roughly 9 million characters equivalent to 1500 text pages. 
Instead, CD-ROMs are now widely used. These contain up to 660 million 
characters (the equivalent of 30 complete copies of the Oxford Textbook of 
Medicine) , cannot be altered, and are physically more robust than floppy 



disks. One of the knowledge bases available on CD-ROM (medical librarians 
usually keep a catalogue) is OncoDisk: this includes an oncology textbook, 
the Physicians' Data Query, and a catalogue of cancer therapy protocols 
known to the US National Cancer Institute, with details of patient 
selection, monitoring, and treatment, and even the names of physicians 
using the protocol. Some physicians already carry a selection of such 
knowledge bases with a portable CD-ROM reader and a notebook computer, to 
help them answer questions about patient management. 

To keep a knowledge base current, updates must be sent out — a major 
enterprise if more than three or four a year are needed or the numbe of 
users is large. One solution is to allow access to a copy held and updated 
at a central point. An example is the Edinburgh Poisons Information System, 
which holds chemical and clinical data on thousands of poisons. [5] Access 
is gained through a Prestel terminal connected to a telephone line, and the 
system currently handles over 40 000 inquiries a year. Disadvantages of 
such dial-up knowledge bases are the need for special, sometimes arcane, 
search languages, and the slowness and cost of telephone links. 

To allow wider access to centrally held knowledge bases, there are 
proposals by the European Community and others for "open systems 
interconnection". Establishment of an electronic "global village" will 
require standardisation of the codes used for clinical terms: but linkage 
of heterogeneous medical nomenclatures such as the ICD-10, SNOMED, and Read 
clinical coding systems, even through the "Unified Medical Language 
System", [6] has yet to be achieved. The physical linking of computer 
systems is easier, with fast cable, fibreoptic, or cellular phone links 
(which allow portable computers to communicate with their base [7] ) . 

Teaching aids 

Teaching aids likewise include a medical knowledge base and programs 
to frame and execute searches. However, they are designed to enhance 
knowledge and understanding by presenting facts, explanations, and examples 
at the user's own level, thus superseding the old "teaching machines" that 
subjected all students to the same material. The knowledge base might 
consist of facts about a disease, or differential equations linking 
physiological variables, together with a search program. These are coupled 
with a program that elicits and records the facts a student does not know 
by asking suitable questions. The search program then assembles the unknown 
facts, together with suitable text, to display remedial material, before 
retesting the student. More elaborate systems provide a glossary of terms 
and offer static or video images and feedback through voice synthesisers; 
many use hypertext techniques. 

A slightly different approach is to use high-fidelity patient 
simulations, as in the "Mac" series. [8] 'MacMan', for example, consists of 
linked mathematical models describing the circulatory, respiratory, and 
fluid-balance systems with programs that allow the student to change key 
parameters and assess their effects on the simulated person; the 
mathematics are hidden behind an attractive graphic presentation which 
helps students to link their knowledge of anatomy and physiology and to 
learn about dynamic relations between body systems. [9] Students seem to 
like these commercial teaching programs. [10] 

One reason why the early teaching machines were badly received is 
that isolated or gratuitously offered facts are not well remembered: 
students learn best when they are confronted with a problem, [11] 
especially if they are emotionally involved in its solution. Some of the 
technologies outlined earlier make it easier to simulate cases in realistic 
fashion. For example, it is now possible to include short sections of video 
showing a "patient" in a hospital setting with authentic background noises 
punctuated by a nurse requesting advice, and then give the user guidance 
according to his or her responses to an on-screen question. One annoying 
aspect of these systems is that a typing error or the wrong vocabulary can 
be misconstrued as ignorance. Research is now focusing on more 
"intelligent " tutoring systems . 

The big drawback of all these teaching systems is the vast amount of 
work needed to assemble even a brief lesson, since there must be enough 
material to allow for all possible responses to each question. If the 
medium is interactive, video, this means dozens, sometimes hundreds, of 
video clips, all of which must be filmed in the same location with the same 
actors. Consortia of medical schools and others are forming to build such 
programs, but new dilemmas have arisen such as the need for standards to 



allow interchange of material, and the question of who owns copyright. 
Medical decision aids 

A medical decision aid has a knowledge base and search program 
similar to that of a teaching system, but contains a program that builds a 
patient model rather than a student model. At its simplest, this program 
may ask the user to type in responses to questions about a patient such as 
age, height, weight, and lung function results, and then formulates a 
suitable search of its knowledge base . In this example, the knowledge 
base contains formulae to predict normal lung function test results and 
information about what abnormal results mean. [12] Instead of symbols 
representing medical details or strategy articulated by expert clinicians, 
the knowledge base of some systems contains a mathematical distillation 
of a series of cases with known outcomes, such as the prior probabilities 
of diseases and the conditional probabilities of certain findings in 
patients with those diseases . [13] The programs that build the patient 
model and search the knowledge base comprise a "reasoner", which 
reconciles the clinical features of a patient logically or mathematically 
with the knowledge in the system, to produce inferences. Such inferences 
might include a list of possible diagnoses with their probabilities, [14] 
potential adverse drug reactions, [15] or the therapy recommended by an 
oncology protocol. [13] 

The first computerised medical decision aid was built in the late 
1950s, [16] and thousands have been made since then. They are best 
classified by their clinical role. One type interprets patient 
data — sounding an alarm, reporting on an ECG, [17] or highlighting abnormal 
laboratory test results. [12] The second kind requests the user to enter 
patient data, and proposes a differential diagnosis [18] or management 
plan. Decision aids of this sort are the commonest today, but they resemble 
a Greek oracle in their reluctance to allow their knowledge to be used for 
other purposes. [19] The third type allows a more flexible interaction, 
with mo re sophisticated search and model— building programs, so that user 
and system can negotiate jointly for solutions. [20,21] 

Most medical decision aids use one of three reasoning methods — Bayes ' 
theorem, decision theory, or symbolic reasoning or "artificial 
intelligence" techniques. Bayes* theorem tells us how to use probabilities 
derived from a database of past cases with a known outcome to calculate the 
probability of the outcome in an individual case. [14] Decision theory is 
an extension to Bayes* theorem that lets us add weightings representing the 
value of each possible outcome to a decision tree, to calculate the 
"expected utility". [22] Symbolic reasoning methods include the use of "if 
. . . then" rules and other complex computer models [23] to link clinical 
findings with their causes and management. All these reasoning methods have 
been refined since their introduction and are now being combined--f or 
example, in directed acyclic graphs, [24] one of which [25] is shown in 
fig. 2. Other reasoning methods include obscure computational models 
inspired by the structure of brain. [26] These "neural nets" rely on a 
process called machine-learning, and their evaluation is far from easy. 
[27] 

The patient simulators used by students can also be helpful to 
clinicians. For example, a program that predicts blood glucose profile over 
24 hours with a specific insulin regimen [28] may allow a doctor to explore 
alternative managements for a patient in a series of computerised 
experiments that hold no risk for the patient. 

Few of the developers of medical decision aids see them as anything 
other than animated reference books, complementing rather than replacing 
human knowledge. Not so Weed with his "Problem-Knowledge Coupler". [29] A 
coupler is a computer program designed to provide clinicians with all the 
medical details they need to tackle a specific clinical problem, without 
committing medical facts to memory. Couplers are available for some 15 
common conditions at present: if widely adopted, such systems might have a 
revolutionary impact on medical education, but would leave their authors in 
a position of unique responsibility. 

Medical decision aids have already proved themselves clinically, with 
the Leeds Abdominal Pain System halving perforated-appendix rates in a 
multi-centre study, [30] Pozen's chest pain system leading to a reduction 
in inappropriate admissions to a coronary care unit, [31] and ONCOCIN (a 
symbolic reasoner for assisting in protocol-driven care) improving the 
completeness of data collection in cancer patients. [32] Systems are being 



built to address a clinical need, [33] but may raise complex medicolegal 
questions for their users and those who supply them. [34] However, there 
are also more immediate dilemmas for users and developers. 
Limited scope of decision aids 

No medical decision aid contains as much knowledge as a medical 
textbooks, and the intellectual and practical challenges of building and 
maintaining large knowledge bases to support doctors remain daunting. 
Although it is possible to express lower-level medical details in knowledge 
bases, making higher-level knowledge explicit is a complex "knowledge 
engineering" task that requires understanding about how to acquire, 
represent, and reason with knowledge. [23] There are similar problems with 
building large databases of patient cases and extracting summary measures 
from them to reason about new patients, who may not be drawn from a similar 
population. [35] 

Some decision aids, however, have achieved good coverage of medicine. 
The INTERNIST system performed acceptably in diagnosis of a range of 
clinico-pathological cases from the New England Journal of Medicine. [18] 
It is now commercially available as *QMR' on personal computers, covering 
over 600 diseases and 4300 patient findings. DXPlain is another 
broad-ranging system that was widely available through a computer network. 
[36] In Britain, the 'Oxford System of Medicine* is planned as a 
comprehensive flexible information system for general practitioners, and a 
prototype containing a fraction of the required knowledge has demonstrated 
that the project is technically feasible. [21] One alternative to building 
a single large knowledge base and then adding a reasoner is to 
integrate several different decision aids, perhaps even using different 
reasoning methods, into one flexible resource, such as the EXPLORER system 
built at Harvard. [3] 

Difficulties of communicating with computers 

Many people become frustrated when trying to persuade computers to do 
what is wanted. When a doctor feeds patient data into a decision aid, a 
good system will not only communicate its advice back but also convince the 
user that the advice is well based. With the *QMR* system it took an 
experienced user between one and four hours to enter enough patient data to 
generate a useful differential diagnosis. [37] 

Most decision aids communicate only by text. * IntelliPath * , however, 
includes a laser videodisk storing 5000 photomicrographs. [38] These are 
used to augment text definitions of histological features and to show which 
features best distinguish between possible diagnoses. Some 300 copies of 
the system, which is supported by the American Society of Clinical 
Pathology, have been distributed. Other methods that may assist in rapid 
and accurate data entry include graphic "user interfaces" (fig 3), 
handwriting and speech recognition, [39] and direct interviewing of the 
patient by computer. [14] The ideal, however, is to integrate decision aids 
with an electronic medical record, so that all relevant data are then 
available, as with the HELP system. [40] 

How do decision aids explain their advice to the physician? One 
system allocates "weights of evidence" to each clinical finding; [14] 
others use their symbolic knowledge about anatomy, pathophysiology, or 
mechanisms of drug action to generate textual explanations of their 
reasoning; [23] yet others are able to explain not only how they arrived at 
their advice, but also how it would have differed had certain patient 
features been absent, or why an alternative explanation is implausible. 
However, as with intelligent tutoring systems, it is often difficult for 
the system to understand what kind of explanation is required. 

Need for evaluation 

A major concern is that computers, like doctors, are often unable to 
admit their ignorance. They also have enormous placebo power, exacerbated 
by the term "expert system", which may cause inexperienced doctors to 
accept their advice without question. Before any system is used to support 
patient care, we must be confident that it is giving sensible advice. The 
content of a decision aid's knowledge base, the rigour of its reasoning, 
and its performance on suitable test cases must all be examined. [41] Even 
if the decision aid's performance is promising, it may still lack 
beneficial impact if it is difficult to use or gives implausible advice: 
[42] as with drugs, controlled clinical trials are the only way to assess 
their impact on doctors and patients. [41] 

An ideal knowledge system 



What every doctor needs is a portable, accurate, and adaptable source 
of medical knowledge. At present, nothing of the kind exists; but it is not 
too soon to think about the features required of such a system. What 
follows is based partly on the surveys of doctors discussed in the first 
paper. [43-45] 

The system must be comprehensive 

The system should cover the majority of diseases, therapies, and 
investigations as well as basic medical sciences, and should integrate 
text, graphics, moving images, and sound. 

Knowledge contained in the system must be current, 

accurate, and verifiable 

The certainty of the details stored should be stated, together with 
their sources to allow easy verification. The system should maintain its 
own record of the age of its knowledge, and updates should be issued 
regularly; all information should be safeguarded against unauthorised 
modification. To avoid ambiguity and to ease maintenance, a consistent 
vocabulary should be used, possibly even a logical language, [21] 

Knowledge in the system must be easily accessible, 

where doctors see patients 

The system should be portable and unobtrusive — preferably fitting 
into a white-coat pocket — and also cheap, indestructible, and workable 
without special equipment. To assist in clinical decision making, it must 
be fast and easy to use. Instead of a typewriter keyboard which is bulky 
and slow, the system should incorporate hand-writing recognition (like the 
new generation of pen-input computers) . [4 6] The indexing system should 
include abbreviations and synonyms used in clinical practice; cross 
references to MeSH would facilitate literature searches. To help users 
absorb new material, updates should include digestible text overviews, 
analogous to the reviews and editorials in the medical weeklies. 

The system should be adaptable 

Since clinicians will be using the system throughout their working 
day, it should accommodate electronic notes and be adaptable to their own 
preferred terms or abbreviations. This could be achieved if each' doctor 
carried a personal electronic badge which broadcast their identity and 
other data to the system in their hands. [7] The system should also be able 
to adapt itself to specific clinical problems, tailoring its reports and 
advice according to patient features. To do this, it could be linked by 
cellular radio networks to clinical data systems. This would also help it 
to predict, from information on who was using it and where, what knowledge 
the physician might need next-- "anticipatory computing". [47] 

Conclusions 

There are few systems that meet even one of the above criteria, but 
advances in information technology will make the ideal knowledge system 
technically feasible. Furthermore, developments in medicine itself, and in 
the administrative framework surrounding it, suggest that the intellectual 
and practical challenges will be met. 

One development' is that medical societies are taking increasing 
responsibility for their members' knowledge as well as their professional 
competence. [48] Thus, specialty societies are defining management 
protocols, minimum data sets, and the content of coding systems. However, 
establishing and maintaining a fund of medical knowledge is a major 
undertaking, and few specialty bodies will have the resourced to do it. 

Another development is the establishment of medical informatics as 
the discipline concerned specifically with the acquisition, storage, and 
communication of medical knowledge and data. [49,50] Training courses, 
academic departments, professional journals, and societies now exist, [51] 
together with a definitive textbook . [ 52 ] The big issues in medical 
informatics include how to represent and store medical knowledge 
electronically and how to help doctors extract knowledge from these stores 
and apply it to their patients. Collaboration with specialists in medical 
education, computer science, and medical cognitive science [53] will be 
necessary to resolve some of these questions. A further challenge is to 
find ways of evaluating and comparing the sources of knowledge, and of 
assessing their impact on medical decisions (and on patients) . 

If the medical profession could harness the medical, technical, and 
managerial skills and produce a system such as that described above, the 
quality of medical care could take a quantum leap forward. Some clinicians 
may argue that reliance on such support will degrade their skills. This was 
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the cry when Laennec introduced the stethoscope. 

These two papers were written partly at the National Heart & Lung 
Institute, London, and partly while I held an MRC travelling fellowship at 
the Section on Medical Informatics, Stanford University. I gratefully 
acknowledge the assistance of colleagues at both sites, others who have 
contributed their time, and the staff of the Lister Hill National Center 
for Biomedical Communications, US National Library of Medicine. 
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ABSTRACT: DXplain is an evolving computer-based diagnostic 
decision-support system designed for use by the physician who has no 
computer expertise. DXplain accepts a list of clinical manifestations and 
then proposes diagnostic hypotheses. The program explains and justifies its 
interpretations and provides access to a knowledge base concerning the 
differential diagnosis of the signs and symptoms. DXplain was developed 
with the support and cooperation of the American Medical Association. The 
system is distributed to the medical community through AMA/NET -- a 
nationwide computer communications network sponsored by the American 
Medical Association — and through the Massachusetts General Hospital 
Continuing Education Network. A key element in the distribution of DXplain 
is the planned collaboration with its physician-users whose comments, 
criticisms, and suggestions will play an important role in modifying and 
enhancing the knowledge base. 

SHORTLIFFE (Ref. 1) defines a computer-based medical decision-support 
system as a computer program designed to help health professionals make 
clinical decisions. Under this broad definition, there are many 
decision-support systems in use today. They include such applications as 
clinical laboratory systems, pharmacy systems, and radiology reporting 
systems. For the most part, these are "passive"- information management 
tools. That is, they make information more readily available to the 
physician but they do not provide "active" decision support by applying 
medical knowledge to a specific patient's data, nor do they recommend a 
specific conclusion or course of action. 

There has been less success in developing active systems. One strategy 
used in such systems applies predefined rules to the patient's data and 
alerts providers to conditions that might require action. The most 
successful system for inpatient care is the HELP system. (Ref. 2) In 
ambulatory medicine, two examples of systems that provide patient-specific 
recommendations for ambulatory care are. CARE (Ref. 3) and COSTAR. (Ref. 4) 
There are also a few examples of "expert" systems that guide the physician 
in treating certain complex, but well-defined, conditions, eg, the ONCOCIN 
system. (Ref. 5) These systems contain in-depth knowledge about a 
well-defined subset of medical knowledge and require considerable and 
detailed information about the patient's clinical status to provide the 
computer-generated consultations . 

Computer-aided diagnosis in general medicine requires a much broader 
level of decision support. There have been a number of provocative research 
efforts in computer-aided diagnosis, (Ref. 6-9) but these have been 
research prototypes whose contributions were primarily methodological. With 
the exception of the work by deDombal et al, (Ref. 10) these programs were 
not made available to the practicing physician. There are a number of 
reasons to explain the limited impact of information technology in the 
diagnostic aspects of medical decision making. (Ref. 1,11,12) Major 



problems have included difficulty in accessing and interacting with the 
systems, a limited area of application (eg, one of the most successful 
computer-aided diagnosis programs is limited to the differential diagnosis 
of the acute abdomen (Ref. 10)), a limited ability of the systems to 
explain and justify the interpretations, and less than acceptable quality 
of the interpretations. Some investigators believe that there are major 
intellectual and technical problems that must be solved before there can be 
a truly reliable consulting program. In particular, it is claimed that 
computer programs must be able to incorporate pathophysiological knowledge 
and causal relationship based on pathophysiological reasoning. (Ref. 12) 

We accept the validity of these concerns, but take a more optimistic 
position concerning the potential for using currently available knowledge 
and technology to assist the physician in the information needs of daily 
practice. A recent study suggests that the information needs of physicians 
in office practice are not being met by printed sources such as textbooks 
and journal articles. (Ref. 13) Of the practice-related questions that were 
identified, 25% involved the diagnosis of symptoms, physical findings, or 
syndromes. We believe that a significant portion of the practical needs of 
clinical medicine can be met now by providing ready access to a 
computer-stored knowledge base of diagnoses and their associated signs and 
symptoms . 

We believe it is possible to develop a computer program that uses an 
extensive knowledge base and relatively simple computational models to 
provide significant diagnostic problem-solving assistance to the practicing 
physician. The program should allow the computer-naive user to enter a set 
of signs and symptoms using a typical medical vocabulary. The program would 
then generate a list of hypotheses that deserve consideration, comment on 
the diagnostic relevance of each sign and symptom, and suggest specific 
additional data elements that might clarify the differential diagnosis 
currently favored by the computer model. (Ref. 9,14-16) 

This article describes DXplain, an evolving computer-based diagnostic 
decision-support system designed for use by the physician who has no 
computer expertise. (Ref, 14) DXplain accepts a list of clinical 
manifestations and then proposes diagnostic hypotheses. DXplain explains 
and justifies its interpretations and provides easy access to a 
comprehensive knowledge base concerning the differential diagnosis of the 
set of signs and symptoms . 

DXplain . does not attempt to make a single diagnosis to mimic the 
behavior or replace the judgment of the expert clinician. DXplain has a 
less ambitious, but perhaps more attainable goal: to suggest a list of 
diagnoses that should be considered given a particular set of signs and 
symptoms. Because DXplain is in a continuing state of development, we have 
labeled it an evolving diagnostic decision-support system. Our plan is that 
DXplain will continue to improve through collaboration with its 
physician-users whose comments, criticisms, and suggestions will play an 
important role in modifying and enhancing the knowledge base and the 
algorithms used in the computer-generated interpretations. 

SYSTEM CAPABILITIES 

We believe that the following six criteria are important for a • 
differential diagnosis decision-support system intended for use in routine 
clinical practice. The system should (1) be easy to use by physicians who 
have little or no computer background; (2) be based on comprehensive 
medical content; (3) provide correct and accurate interpretations; (4) 
justify its interpretations; (5) be convenient to access from the 
physician's office, hospital, or home; and (6) evolve and improve as a 
result of user criticism and analysis of user sessions. 

Easy to Use 

It has been the general experience that few practicing physicians will 
use computer programs requiring extensive training or knowledge of computer 
technology. DXplain demands little knowledge of computer technology and 
requires onLy the use of the vocabulary common in medical practice. On-line 
help is available for an explanation of the system commands. 

As with any decision-support system, DXplain uses a controlled 
vocabulary for communicating patient information to the system. Computer 
technology has not yet progressed to the stage in which a computer program 
can recognize the free-form narrative text that a physician might use to 



describe clinical manifestations in a medical record or with a colleague. 
The predefined medical vocabulary used by DXplain consists of more than 
4 700 terms based on the clinical and basic laboratory data that might be 
collected in an ambulatory practice or in the emergency department. 

DXplain has several features that assist the user in selecting the 
desired terms from the controlled vocabulary. For instance, there is 
extensive synonym and abbreviation terminology; also, the system can 
recognize and correct many misspellings. DXplain can recognize terms that 
are "close" (eg, "congestive heart failure" and "heart failure"), as well 
as synonyms for terms at both the single-word level (eg, "kidney" and 
"renal" are equivalent in many different terms) and at the fullphrase level 
(eg, "anisocoria" is equivalent to "pupillary inequality," "factor VIII 
deficiency" is considered equivalent to "hemophilia"; and "blood glucose 
elevated" is considered equivalent to "hyperglycemia"). In addition, many 
abbreviations are recognized (eg, "ESR" and "CHF") to speed data entry. 

The DXplain vocabulary structure is hierarchical, which allows 
flexibility in specifying the precise level of detail of the clinical 
manifestation. Thus, DXplain groups more specific terms (such as "lower 
abdominal pain") under a less specific term (such as "abdominal pain"). 
This hierarchy is important for both the user interface and the algorithm 
used for interpretation. When a user enters a specific term, such as "right 
lower quadrant pain, " all of the appropriate less-specific terms, such as 
"lower abdominal pain" and "abdominal pain, " are assumed by the system, but 
related terms at the same level of specificity, such as "left lower 
quadrant pain, " are not assumed. 

A typical user interaction with DXplain is illustrated in Fig 1. 
Examples of how the system helps the user select the appropriate terms 
during input are illustrated in Fig 2. 

A second important factor relating to ease of use is the rapidity with 
which the user can enter the clinical manifestations and extract the 
desired information and interpretation. The system is designed to be 
largely self-explanatory. There is no need to read manuals: the beginning 
user can take advantage of menu selection; the experienced user can make 
use of an abbreviated command language to speed the interaction. The 
requirement for rapid time response was a critical design factor that 
influenced decisions made about the organization of the knowledge base in 
the computer. It requires about two minutes to complete the dial-in 
sequence to log on to AMA/NET and to connect to the computer located at 
Massachusetts General Hospital. The entry of the seven terms illustrated in 
Fig 1 required about one minute. Evaluation of the clinical data by the 
computer . program and presentation of the interpretation and list of 
diagnostic hypotheses takes from 10 to 20 s. 

Accurate and Comprehensive Medical Content 

The development of a comprehensive knowledge base in medicine, whether 
printed or computer based, is a large effort. The merit of any 
decision-support system depends to a significant extent on the quality of 
the knowledge base used in formulating its recommendations. One aspect of 
any knowledge base, whether printed or computer based, is the inability of 
the author to certify that the entire knowledge base is totally accurate 
and comprehensive. 

There are no methods to extract automatically the relevant information 
from the published literature. In fact, the published literature is not as 
useful as desired since it often does not provide the quantitative 
relatrionships between clinical manifestations and diseases that are 
required for a decision-support system. For example, one frequently finds 
statements like "symptom X is 'occasionally found' in the disease" or "it 
is *not uncommon' to have symptom Y" or "the 'great majority' of the 
patients with the disease will demonstrate symptom Z." 

To provide the necessary knowledge base for DXplain, we begin with the 
computer-based version of Current Medical Information and Terminology 
(CMIT) , which is published and supported by the American Medical 
Association. (Ref. 17) Current Medical Information and Terminology contains 
summaries of information concerning the etiology, signs and symptoms, 
laboratory findings, and disease course for more than 3000 distinct 
diseases, disorders, and conditions. 

To transform this knowledge base into one suitable for a 
decision-support system, it was necessary to carry out a considerable 



amount of manipulation and additional content acquisition. For the disease 
list for DXplain, we combined some diagnoses from CMIT to form more 
inclusive definitions, while others were subdivided into more specific 
forms of the disease (eg, separate stages of a disease, forms of the 
disease specific to certain demographic groups, and forms attributed to 
different etiologic agents) . Additional diseases were added when 
deficiencies in CMIT were noted. 

As the first step in specifying the controlled vocabulary of terms to 
represent the clinical features of each disease, terms were extracted from 
the text of CMIT, yielding a list of some 6000 words and phrases. An 
extensive review of this list was undertaken to lend coherence to the 
vocabulary. Synonyms were merged where appropriate and similar terms were 
linked together in a hierarchical manner to express their relationships 
(usually in terms of such qualifiers as severity, duration, or anatomic 
location). The outcome was a term directory consisting of 4000 descriptors. 
As disease descriptions were compiled, 700 additional concepts were added 
to the directory. 

We used a three-step process to generate the disease-term 
relationships of the knowledge base. First, a list was compiled from the 
CMIT data base and from expert judgment of all the terms that had some role 
in either supporting or ruling out a particular disease. Next, medical 
texts were reviewed to determine the frequency of each clinical finding in 
the disease. Where necessary, in circumstances of recent discoveries or 
obscure conditions, literature searches were conducted to supplement 
standard references. Finally, an estimate of the potential for the presence 
or absence of each term to evoke or refute the diagnosis was made. 

This content development was an iterative process that was done by the 
system developers in collaboration with 13 physicians representing several 
medical disciplines. Because of the number of individuals involved, 
extensive discussions were required to reach a consensus on content and 
assignment of appropriate weighting factors for the relationship of terms 
to diseases. Once guidelines were established, authors were trained in 
filling out work sheets for each disease. Each work sheet included all 
terms that had been previously assigned, a prescribed set of demographic 
terms (age, sex, and duration of symptoms), and a generous number of blank 
lines for adding new terms. After the work sheets were completed, one of us 
(J.J.C.) reviewed them to ensure consistency of content, terminology, and 
assignment of evoking potentials. 

The result of this work is DXplain' s knowledge base consisting of 
descriptions of approximately 2000 diseases, about 4700 terms (signs, 
symptoms, and more) , and some 65 000 relationships among them. On average, 
each disease description contains relationships to 35 terms. Each* 
relationship identifies the frequency of the manifestation among patients 
with the disease and the strength with which a manifestation evokes a 
diagnosis for consideration. 

From a practical viewpoint, it is impossible to verify completely all 
the elements in a knowledge base that is the size and complexity of 
DXplain. We believe that there must be a continuing effort to detect and 
correct incomplete or inaccurate disease descriptions. In addition, the 
knowledge base can never be static, but must continue to evolve as new 
knowledge is gained and new evidence is identified about each disease. 

One of the major advantages of having the knowledge base reside on a 
single central computer is that the knowledge base can be updated easily 
and as often as necessary. The addition of a new term, disease, or synonym, 
the modification of the descriptor-disease relationship, or the enhancement 
of the program can be done quickly. Because these changes are immediately 
available to all users, the system can be dynamically responsive to user 
suggestions concerning deficiencies and improvements . 

Provide Correct and Accurate Interpretations 

In designing a differential diagnosis decision- support system, we 
assumed that the most useful assistance that can be provided to the 
practicing physician is to suggested the reasonable diagnoses that should 
be considered, given any particular set of signs and symptoms. We believe 
that in many complex diagnostic problems (and many not so complex) , a major 
reason for not making the correct diagnosis is the failure of the physician 
to generate a differential diagnosis list that is comprehensive enough to 
include the correct diagnosis, yet sufficiently focused to include only the 



more likely diagnostic possibilities. 

The goal in DXplain is to assist the physician in two phases of the 
process of differential diagnosis: (1) to remind the physician of diseases 
that should be considered as possible candidates to explain the patient's 
condition and (2) to provide information about specific diseases that might 
be eliminated from consideration. We explicitly reject the objective of 
determining the "correct" diagnosis. We reason that the clinician will 
always have a more complete picture of the patient than will be entered 
into the computer program and that a more realistic goal for a 
decision-support system is to bring to the user's attention the plausible 
explanations for a patient's signs and symptoms. The user may then consider 
which diseases are appropriate to the case by applying common sense, 
clinical experience, and full knowledge about the patient and, possibly, by 
further interacting with the program. 

DXplain is not intended to aid the specialist working in his/her 
specialty area. Thus, the cardiologist dealing with a patient with a 
complex murmur is unlikely to find DXplain very helpful; but the 
cardiologist dealing with a patient with abdominal pain may obtain useful 
assistance. The most obvious way in which DXplain can help is in suggesting 
obscure or rare diseases that may be rarely seen by most physicians. 
DXplain also may be helpful by suggesting diseases that present in atypical 
ways , 

DXplain uses a relatively straightforward algorithm to select its list 
of plausible diagnoses. This algorithm has been described in a previous 
publication. (Ref. 14) The DXplain selection rule takes advantage of 
conditional probabilities and a scoring system similar to a Bayesian 
computation. Models such as this have been frequently used in other 
diagnostic decision-support projects, the most influential being the 
ranking algorithm used in INTERNIST/QMR. (Ref. 7,14) Our development of 
DXplain was strongly influenced by the experiences and the limitations of 
the original efforts of Miller and colleagues with INTERNIST, and their 
later development of QMR. 

The knowledge base for DXplain contains more than 65 000 relationships 
between diseases and patient descriptors (or clinical manifestations) . The 
format for these relationships and the data structures are similar to the 
format and data structures used in the INTERNIST/QMR system (Ref. 7) except 
that in DXplain terms cannot only support a given diagnosis, but can also 
contradict other diagnoses. The relationships in DXplain are concerned with 
three different elements: (1) term importance, (2) term frequency, and (3) 
term-evoking power. 

Term importance is used to express how significant the particular term 
is in indicating the presence of disease. A high term importance is given 
to findings that can be identified with high reliability or are rarely 
found in healthy persons and, therefore, should be explained by some 
disease within the differential diagnoses. 

Term frequency is used to identify how often a particular term is 
expected to occur in a specific disease. The possible values for term 
frequency can be one of seven different states ranging from "always" to 
"never . " 

Term-evoking power is used to identify how strongly a particular term 
supports the possibility that a specific disease might be present. This 
term is related to the concept of predictive value positive (the predictive 
value of a positive test — this is the probability of a disease being 
present given the presence of a certain finding) . The possible values for 
term-evoking power can be one of eight different states ranging from 
"certainly supported" to "weakly supported" and from "weakly contradicted" 
to "strongly contradicted." 

DXplain uses the numerical values of these different relationships to 
derive the list of the diagnoses that should be considered (Fig 1) . DXplain 
selects diseases to be included on this list on the basis of how well the 
manifestations of a particular disease match the descriptors that have been 
entered by the user. The diseases are presented to the user in two lists: 
"common diseases" and "rare diseases"; in addition, a serious disease (one 
that may require relatively immediate action) is so indicated on the list 
by an asterisk. 

The user can request DXplain to explain why any specific diagnosis was 
included (Fig 3) . DXplain will present the clinical findings entered by the 
user that support the selection of that disease, the clinical findings that 



would not be expected in that disease, and additional clinical findings 
that would be expected if that disease was present. In this way, DXplain 
assists the user in understanding the logic used by the program and 
facilitates pattern matching by the user in comparing DXplain 's disease 
description with his/her knowledge about the patient . The intent is to 
present sufficient information so that the user can always use has/her own 
clinical judgment as to the appropriateness of DXplain's interpretation. 

The user can ask DXplain to consider a specific diagnosis that was not 
included on the initial list (Fig 3) . DXplain will then present the same 
analysis, as described in the previous paragraph, for the diagnosis under 
consideration and, furthermore, will include this diagnosis in any later 
interpretation after additional findings are entered. 

The user can change DXplain into an interrogative mode wherein the 
system will question the user about the presence or absence of significant 
findings that have the potential for clarifying DXplain *s current 
differential diagnoses (Fig 4). This mode is particularly useful in helping 
the user select the clinical manifestations that are important without 
forcing them to enter a large number of less relevant findings. At any time 
in this mode, the user can interrupt DXplain to ask "Why?" ie, to ask 
DXplain to justify why this particular clinical manifestation is important. 
DXplain will respond by displaying the name of the disease that is being 
considered at that point in the interaction and the reason the particular 
finding might be important in conforming the presence of that disease. 

DXplain 's ability to explain and justify are key elements of the 
system. It is critical that this system not be perceived as a magic black 
box that can somehow provide the "answer" to a complex diagnostic problem. 
We believe that physicians will not accept DXplain as a useful diagnostic 
assistant unless the clinical interpretations seem reasonable and unless 
the system can offer explanations that are understandable and persuasive. 
(Ref. 18) 

Convenient to Access 

Almost a decade ago, Shortliffe wrote: "A recurring observation as one 
reviews the literature of computer-based medical decision making is that 
essentially none of the systems has been effectively utilized outside of a 
research environment, even when its performance has been shown to be 
excellent." (Ref, 8) For the most part, this observation is still true 
today. One important barrier is that the practicing physician cannot easily 
access a computer-based decision-support system from his/her office, or any 
other location, at any time. There are a number of systems that are 
available in university hospitals and that are of great importance in the 
host institution and have considerable value as demonstration models. 
However, none of these systems provide support to the practicing physician 
on a national scale. 

DXplain is unique in that the decision-support capability is easily 
accessible using only a simple computer terminal (a microcomputer can also 
be used to make the connection) , a telephone modem, and a telephone call 
(usually a local number in most of the major cities of the United States) . 
DXplain is also available in a similar fashion in Canada and Japan. There 
is no start-up cost associated with purchase of the programs or of the 
knowledge base. Many physicians already have the necessary technology to 
access DXplain since the needs are the same as those used for on-line 
bibliographic search services. 

DXplain was developed with the support of the American Medical 
Association and is designed to be distributed to the medical community 
through AMA/NET -- a nationwide computer communications network sponsored 
by the American Medical Association. Physicians and other professionals can 
access the system through the AMA/NET . Medical schools and teaching 
hospitals can access the system either through AMA/NET or through the 
Massachusetts General Hospital Continuing Education Network. In both cases, 
the cost of accessing DXplain is directly dependent on the length of time 
one is connected to the system. 

A subscriber to AMA/NET can also access information data bases 
(EMPIRES clinical reference citations. Medical Procedure Coding and 
Nomenclature, the Associated Press Medical News Service, and more), public 
information services (Centers for Disease Control Information Service, 
National Library of Medicine/National Institutes of Health Information 
Services, Adverse Drug Reaction Reporting Form, and more) , electronic 



mail, and the Massachusetts General Hospital interactive medical education 
courses (Hoffer et al (Ref. 19)). AMA/NET also provides documentation and 
telephone support to its subscribers. (For information on AMA/NET, call 
1-800-426-287 3; for information on the Massachusetts General Hospital 
Continuing Education Network, call 617-726-3950.) 

Evolve and Improve as a Result of User Criticism 

A key element in the distribution of DXplain is an integrated 
electronic mail capability. At any point in the interaction a user may 
enter a comment or question into the computer system. This electronic mail 
is read at frequent intervals by the system developers at Massachusetts 
General Hospital and responded to as appropriate. 

We view the plan for the continuing improvement and enhancement of 
DXplain as one of the more important aspects of its development. The major 
potential weakness of any diagnostic decision-support system such as 
DXplain is the quality and completeness of the underlying knowledge base. 
Evaluating a clinical decision-support system is difficult, both 
conceptually and in practice . (Ref . 1,11) Systematic clinical trials or 
formal outcome studies on the impact either of computer-based knowledge 
bases or of medical textbooks are logistically almost impossible. DXplain 
has been used by physicians for over 500 hours at more than 40 different 
test sites in the United States, Canada, and Japan. The initial user 
acceptance and peer review has been favorable, although the evaluation has 
been largely anecdotal. 

The continuing refinement of DXplain will be most fruitful if the 
planned collaboration between the physician-users and the developers 
materializes. We expect and need the participation of physicians who will 
challenge the system with rare diseases and with uncommon manifestations of 
common diseases. The continuing critical review by DXplain users of the 
knowledge base and interpretations of the system will provide important 
feedback in the iterative process of knowledge base development. 

SYSTEM LIMITATIONS 

DXplain has a knowledge base that covers more diseases than are 
discussed in most textbooks of medicine , but in some areas DXplain is 
incomplete, eg, there is only limited coverage of dermatologic diseases , 
where diagnosis often depends on the visual appearance of the lesion. At 
present, there' is only minimal coverage of diseases from psychiatry and 
orthopedics. DXplain presently allows the entry of only a limited set of 
laboratory test findings. The original design goal of DXplain was primarily 
focused on ambulatory medicine ; as a result, the current version of 
DXplain does not allow the entry of many of the complex laboratory tests 
that are performed only in hospitals. We are continuing to add both 
diseases and terms, including the most common laboratory abnormalities. 

DXplain can cope in only a limited fashion with the variations in the 
way that a disease can present based on its evolution over time, degree of 
severity, and the modifications introduced by therapy. In addition, DXplain 
does not identify complex disease patterns caused by the presence of two or 
more diseases in the same patient. DXplain considers each disease and its 
expected manifestations as unique entities. DXplain is unable to recognize 
how the manifestations of one disease can be modified by the presence of a 
second interacting disease. However, DXplain will attempt to select and 
present to the user all the individual diseases that might account for the 
more important findings so that the physician can use clinical judgment to 
carry out any appropriate recognition of disease patterns. 

A number of authors have emphasized the importance of an explanation 
capability to' encourage physicians to use decision-support systems. (Ref. 
1,18) DXplain has an explanatory capability, but it is limited to the 
justification of why a particular disease should be considered (or ruled 
out), based solely on the likelihood of occurrence of the specific clinical 
manifestations in that disease. DXplain has no pathophysiological or 
anatomic knowledge and no ability to consider pathophysiological or 
anatomic reasoning. 

The mere existence of data in a knowledge bank is, of itself, no 
guarantee of completeness and accuracy. The same professional judgment and 
critical appraisal are required when using DXplain as required when reading 
a medical textbook or discussing a patient case. In fact, since the 
computer program does not possess the depth of medical knowledge, the 



wisdom of medical experience, or the ability of a colleague to reason, it 
would be wise to be even more critical of the computer's interpretation. 
One of the weaknesses that is common to every computer-based 
decision-support system is a lack of "common sense" and a relative 
inability to consider important personal, social, family, and employment 
factors of the particular patient. This deficiency is well illustrated by 
an anecdote in the conclusion of Shortliffe*s article. (Ref. 1) 

We believe it is critical that the physician retain the ultimate 
responsibility for identifying the correct diagnosis or diagnoses in any 
given patient. Using DXplain should be considered similar to consulting a 
medical textbook or journal article. DXplain should be used only as an 
adjunct, an information base, and a well-specified medical knowledge 
resource; DXplain cannot be a replacement for the clinician's knowledge and 
experience . 

CONCLUSIONS 

The potential contributions of computer-based decision-support systems 
are based on several factors: (1) the increasing complexity and scope of 
the medical knowledge base, (2) the increasing fragmentation and 
specialization of medical practice, (3) the increasing availability and 
af f ordability of powerful computer technology, and (4) the increasing 
willingness among physicians to utilize computer technology in all phases 
of patient care activity. 

DXplain is an evolving computer system that uses an extensive 
knowledge base and relatively simple computational models to provide 
significant diagnostic problem-solving assistance to the practicing 
physician. The program allows the computer-naive user to enter a set of 
patient signs and symptoms and then generates a list of hypotheses that 
deserve consideration. The system also comments on the diagnostic relevance 
of each sign and symptom and suggests specific additional data elements 
that might clarify the differential diagnosis currently favored by the 
computer model. The advantages of a dynamic, interactive, evolving 
reference tool, such as DXplain, over static, passive textbooks and journal 
articles are exciting. 

DXplain is unique in being a decision-support system that is easily 
and inexpensively available to a large number of physicians through 
nationwide medical information networks. A key element in the distribution 
of DXplain is the planned collaboration with its physician-users whose 
comments, criticisms, and suggestions will play an important role in 
modifying and enhancing the knowledge base and the algorithms used in the 
computer-generated interpretations . 

The most important evaluation of DXplain will be made by the 
physicians themselves as they attempt to use the system in their daily 
practice. As DXplain is used in a variety of practices with a variety of 
information needs, we will learn whether this system is capable of 
providing clinicians with useful information in a timely manner. The 
measure of success is not whether the ultimate diagnosis is listed first or 
fourth in the DXplain list of plausible diagnoses, but whether the 
physician-users perceive that the use of the system has added to their 
understanding of the patient's problems. Through careful analysis of user 
interactions, comments, level of satisfaction, and system performance, we 
expect to learn much about DXplain *s ability to provide assistance in 
making clinical decisions. Based on our experience during the testing 
phase, we are optimistic that DXplain will be a useful educational resource 
and an effective assistant to the physician in daily practice. 
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Computers can never replace physicians. Only physicians have the ability to 
reason logically and to mix that reason with intuition, experience, and 
ethics -- something no machine can ever learn to do." 

Or so many physicians have been telling people proposing 
computer-assisted medical decision making (CMD) since the early 1960s, and 
controversy has continued. But now it looks like the two sides are nearing 
agreement: Computers may never replace physicians, but they can and will 
assist their decisions. 

"Assist " is the key word in CMD. Early advocates tended to 
overestimate the powers of the computer and promised complete clinical 
evaluation at the push of a button. It was natural for physicians to feel 
somewhat threatened. It was also natural that, with experience, such 
promises had to be modified. 

Today there still are only several functioning knowledge-based 
decision systems -- called "knowledge-based" because they make inferences 
using a data base of medical information stored in a computer's memory -- 
and they are extremely modest. More complex systems are currently under 
development, but even these will merely serve as expert consultants; the 
ultimate decision, which requires personal, occasionally subjective 
considerations, still will rest with the practitioner. 

"The early systems attempted to do diagnosis for the doctors -- which 
is crazy," says Francis T. de Dombal, MB, MD, consultant with the Leeds 
Area Health Authority and Reader in Clinical Information Science at the 
University of Leeds, England, "Doctors do some things very well indeed, and 
computers do them very badly. For example, if the Leeds United Football 
Team was playing at home, there 'd be quite a lot of people with bellyaches. 
The experienced doctor would know the source of all these bellyaches, and 
he wouldn't take everyone's appendix out. The computer might well do so." 

de Dombal is well acquainted with the abilities and limitations of 
computers: His Leeds system was one of the first and remains one of the few 
fully operative CMD systems. Able to discriminate between common causes of 
acute abdominal pain, the system has been used for the past eight years in 
three United Kingdom (UK) hospitals, and recently the UK government 
authorized and funded a three-year trial in 11 other hospitals. 

Although both the problem and the possible diagnoses are limited, the 
system has improved diagnostic accuracy for abdominal pain by 20%, 
according to de Dombal. After studying the pattern of admittances in his 
own hospital and others, he and his team constructed probabilities of 
common causes of abdominal pain and entered these into the computer memory 
along with the signs and symptoms associated with each cause. 

'What Have I Got, Doc?' 

When a patient is initially seen with abdominal pain, the examining 
physician records observations on a preprinted form that requests 
information about the pain, other symptoms, medical history, and findings 
from the general examination. The physician then gives the form to a member 
of the computing team, who enters it into the computer by choosing an 
"answer" for each of a series of categories. For example, under the 
category of "relieving factors, " he can choose between "lying still, " 
"vomiting," "antacids," "food," "other," and "nil." Other categories 
include sex, age, site of pain at onset, aggravating factors, and site of 
pain at present . 

Next the computer analyzes the data. Essentially, it compares the 



details of the new case with those of hundreds of similar cases in its 
memory and uses a theory of probabilities to calculate the likelihood of 
various diagnoses. The resulting printout displayed to the physician lists 
all the symptoms and gives the probabilities that the patient has 
appendicitis, non-specific abdominal pain, diverticular disease, perforated 
peptic ulcer, cholecystitis, small bowel obstruction, or pancreatitis. A 
sample printout might suggest a 27.11% chance of appendicitis, a 12.28% 
chance of perforated peptic ulcer, and a 60.60% chance of nonspecific 
abdominal pain; the practitioner can combine this information with his 
clinical impressions to arrive at a diagnosis. 

"I would defend to the death the doctor's right to override what comes 
out of the computer if he thinks it's crazy," stresses de Dombal . "Right 
across the top of all our systems it says, 'Clinical judgment must .take 
precedence; these are only suggestions.'" 

After testing the system from January 1971 to August 1972, de Dombal 
found that preoperative diagnoses made with the computer-aided system 
(using data provided by the registrar) were 91% correct. This was in 
comparison with 42% correct for diagnoses made by unaided admitting 
physicians, 71% correct for diagnoses made by unaided house surgeons (less 
than six months experience), 79% correct diagnoses by unaided registrars 
(between one and two years surgical experience), and 81% correct diagnoses 
by unaided senior clinicians (more than five years surgical experience) . 
These results show one obvious advantage of the system: training 
inexperienced physicians. 

"What we're really trying to do," says de Dombal, "is to get the 
inexperienced doctor ' s level of performance up to that of the senior 
clinicians within a few weeks, as opposed to ten years." 

The improvement figures may be misleading, however, since the 
physician often makes the crucial decision before turning to the computer. 

"Before these trials were run, " says Richard Bruce Friedman, MD, 
associate professor of medicine and human oncology at the University of 
Wisconsin Medical School, Madison, "the investigators had already screened 
out every area but abdominal pain. By the time (Ref. aphysicia) knows it's 
abdominal pain, he calls the surgeon and he gets an x-ray — even without a 
computer. The test is an artificial situation and makes no real difference 
to clinical procedure. 

"Even if - you send this program neck pain, it will still make a 
diagnosis of abdominal pain." 

It seems to make a difference to some physicians, though. With the 
help of the World Organization of Gastroenterology, investigators have 
modified the data base to take geographical variation into account, as 
abdominal pain may have different causes in different areas. And since 
1972, the cost of the program has come down considerably: Once run on a 
large university computer, it now can run on small computers like Pets and 
Apples, which are quite common in hospitals and physicians' offices these 
days. 

Moreover, the US Navy is currently evaluating de Dombal ' s program for 
use on submarines and is adapting the data base to this population. 
According to George Moeller, MD, director of the Behavioral Sciences Branch 
of the US Navy, appendicitis is one of the principal medical problems 
aboard submarines. Although some medical problems can be handled aboard, 
surgery of course requires moving the patient off the submarine . 

"That ' s something we would like to avoid, " says Moeller, "not only 
because it (Ref. movigthepatiet ) would interrupt the mission but because 
it's a dangerous procedure in and of itself." Navy corpsmen could use the 
CMD program to effectively decide when to move the patient. Moeller adds 
that the Navy is also considering using a variation of the program for 
chest pain and major behavioral disorders . 

The only other general type of CMD system currently in use is almost 
the complete opposite of de Dombal ' s knowledge-based system: It covers a 
huge range of medical problems but uses no complicated statistical analyses 
to serve up probabilities of certain conditions or diseases. Such a system 
simply stores and retrieves huge bodies of text, allowing overbur-dened 
practitioners to deal with an "information explosion." 

Handling the 'Information Explosion' 

The National Library of Medicine, Bethesda, Md, for example, has 
developed a prototypical Hepatitis Knowledge Base that is in use. The 



American Medical Association and General Telephone and Electronics (GTE) , 
moreover, will inaugurate the first nationwide electronic medical-health 
information system by late 1982. The program consists of a series of 
different data bases, some of which are closer to completion than others. 

The drug data base in particular seems almost ready, according to Dan 
Harris, director of the Medical Information Network at the AMA. To use this 
data base, the physician might type "Valium" into his office computer 
terminal (which can be hooked up to the GTE network) . He then would receive 
a kind of "menu" that would allow him to choose the kinds of information he 
would like, for example, on contraindications, pharmacokinetics, and drug 
interactions. Depending on the information requested, the computer will 
respond with information within seconds. 

A disease data base, taken largely from the AMA's Current Medical 
Information and Terminology (OMIT), is under consideration now and would 
offer information on signs and symptoms, etiology, synonyms, and laboratory 
and pathology data for a named disease. 

"The computer can give this information more quickly than a reference 
book," contends Harris. "If you're a physician in an emergency room whose 
specialty happens to be surgery, and a patient walks in, you may first have 
to know what drugs he's involved with. If these drugs differ from those you 
normally prescribe, this system can give you what you need to know 
immediately. " 

Cost has been the traditional barrier to computer systems, but with 
increasingly common low-priced terminals, this barrier is on the way out. 
However, collecting a data base often takes years of time: Texts and 
journals have to be combed for information which then has to be 
reorganized; files have to be continually updated. For many experimental 
systems that combine huge data bases with complex programs and lofty 
aspirations, this lag time increases from years to decades. 

"It's easy to get the impression from reading (Ref. joural) papers 
that the day is around the corner when all doctors will be getting expert 
consultation from computers," says Dana Ludwig, MD, of the University of 
California, San Francisco. "That's not really true. Work has been going on 
for 20 years now, and there are very few systems being used in day-to-day 
practice anywhere -- let alone in widespread use." 

On the other hand, because computers don't forget, overlook, tire, or 
equivocate, research is proceeding apace. There are plenty of incomplete 
but promising systems in experimental use, most often by physicians with 
computer science experience of their own. Very generally, these 
knowledge-based systems can be divided into two divergent types : 
numerically based and artificial intelligence (AI) systems. 

Experimental Systems 

Many of the numerically based systems resemble de Dombal ' s in that 
they use a statistical rule like Bayes theorem to make decisions. Bayes 
theorem provides a means for calculating the probability of hypothesis A, 
given the probability of hypothesis B. In medical terms, the theorem can be 
used to calculate the probability of each possible disease given a 
description of a particular case. 

One major problem of these Bayesian systems is that they must make 
assumptions that simply do not hold true in real, medicine . Bayes theorem, 
as generally used, "doesn't conveniently allow for the hypothesis of 
multiple diseases in the same patient," says Ludwig. Moreover, "it doesn't 
conveniently allow the representation of nonindependence of symptoms within 
the disease . " 

In other words, most CMD systems using Bayes theorem define certain 
entities as patient attributes and other entities as disease hypotheses. 
They assume that the patient can have only one disease and that, assuming 
he has it, all the attributes — signs or symptoms — occur independently 
of each other. Ludwig 's INFERNET system tries to avoid these obvious 
oversimplifications by applying an extension of Bayes theorem to a network 
of interconnected diseases, symptoms, and signs. 

"This system, " explains Ludwig, "allows the possibility of 
intermediate states. For example, the disease of interest might be 
myocardial infarction, and it might be causing shortness of breath and high 
pulmonary wedge pressure. But actually it causes shortness of breath and 
pressure through an intermediate stage — congestive heart failure. It's 
this intermediate state that links the two symptoms together." INFERNET can 



take this link into account to give a more flexible and realistic picture 
of probabilities. 

But INFERNET, like most other statistical-inference CMD systems, is 
nowhere near ready for clinical use. Not only must a vast amount of data be 
collected, but probabilities of specific diseases must be calculated — a 
time-consuming and expensive project. Beyond that, the questions remain: Do 
complex statistical calculations really model or surpass medical reasoning 
in terms of efficacy? Can the probabilities cranked out of the computer 
make any difference in patient care? 

Many physicians and computer specialists think not and have turned 
instead to that branch of computer science that attempts to simulate human 
reasoning: AT. Definitions of AI seem almost as plentiful as CMD systems, 
but generally AI can be defined as a type of program that generally 
manipulates ideas and symptoms rather than numbers. 

Artificial Intelligence Systems 

"The only way people would be willing to use (a typical statistical) 
program is if they were willing to turn over the decision to the computer, " 
says Edward H. Shortliffe, MD, PhD, assistant professor of medicine and 
computer science at Stanford University. "And I don't think doctors are 
likely to do that -- ever — for a real patient .... If you run a 
program and it says your patient has systemic lupus, probability .89, then 
what do you do? You have no way of deciding if, in fact, the computer used 
the information properly or improperly. And although the computer may have 
asked very reasonable questions about signs and symptoms, how it came up 
with that number and whether or not you should believe it is another 
issue. " 

Shortliffe and other people working in AI believe that physicians 
should be able to evaluate the computer's reasoning and make conclusions as 
to the validity of its decision. Therefore, instead of building a complex, 
unintuitive formula into the machine, they try to model their systems on 
human cognition. The problem, of course, is that they first must learn how 
human cognition works. 

Part of this problem is being solved in the new field of medical 
decision making. Many CMD researchers — both in AI and other areas — are 
involved in the Society for Medical Decision Making and its international 
journal. One of the theories coming out of recent research, according to de 
Dombal (Nyhus LM: Computers and the surgeon. Surg Annual 1979; 2: 33-57), 
is that when physicians make decisions they acquire information, analyze 
it, and then decide the best course of management for the individual 
patient . 

These decisions can be either "algorithmic" — as when a physician 
draws together many pieces of clinical data to arrive at a diagnosis or 
"heuristic" — as when a physi'cian asks a few questions, quickly thinks of 
a likely diagnosis, and switches to specific questions that will confirm or 
refute that diagnosis. A third type of decision — a more pragmatic one — 
is called "payoff": here the physician asks questions that will help him 
determine patient management . 

According to de Dombal, junior physicians and nonurgent problems 
usually evoke algorithmic decisions, while more senior physicians and 
urgent problems evoke heuristic and payoff approaches. Programs aiming to 
simulate the reasoning of an expert clinician, then, might follow a 
heuristic pathway. 

While building AI systems, researchers discover additional information 
about human cognition, which they add to later versions of their systems. 
Shortliffe worked essentially on a trial-and-error basis for his first 
system, MYCIN, as well as his current system, ONCOCIN.For example, he would 
run MYCIN on what he knew to be a meningitis case and see that the system 
performed poorly. He would then analyze its "reasoning" sequence to see 
where it went wrong and modify the program until it yielded a reasonable 
recommendation . 

According to Shortliffe, trial-and-error is "easier than asking an 
expert for everything he knows about meningitis and writing it down. It's 
very hard for people to analyze their own knowledge." One of ONCOCIN's 
major purposes is to extend basic knowledge about decision making and AI . 

Its other purpose is to develop a clinically useful oncology 
consultation system that can explain its reasoning to the physician. 
ONCOCIN was implemented in 1981 for experimental use in patients with 



lymphoma, and Shortliffe hopes to expand the program to therapies of many 
other cancers. By storing lengthy and complex protocols in the computer 
memory, he hopes to use ONCOCIN to greatly improve patient care and 
eliminate departures form protocol guidelines. 

Like its predecessor MYCIN, ONCOCIN bases its decisions on a series of 
"if-then" rules. Depending on answers to the "if" condition, provided by 
the physician, the computer uses its preprogrammed "then" response as the 
"if" condition for the next rule. An English "translation" of the rule for 
determining "attenuated dose" is: 

RULE 075 

To determine the current attenuated dose for all drugs in MOPP 
(nitrogen mustard, Oncovin, procarbazine, prednisone) or for all drugs in 
PAVE (procarbazine, Adriamycin, and Velban) 

If: (1) This is the start of the first cycle after a cycle was 
aborted, and 

(2) The blood counts do not warrant dose attenuation 

Then: Conclude that the current dose is 75% of the previous dose. 

ONCOCIN also draws on stored data files containing attributes of other 
patients as well as rules about drug usage, tests, and other information 
relevant for protocol decision making. Combining this entire knowledge base 
with information about the new patient *s diagnosis and test results, 
ONCOCIN produces a recommendation, along with an explanation of its 
decision, about five minutes after the physician starts entering data. 

Another experimental system using AI, but for a much larger area of 
medicine, is CADUCEUS (formerly INTERNIST) at the University of Pittsburgh. 
The program, supported by the National Institutes of Health and the 
National Library of Medicine, initially was limited to internal medicine 
but is currently branching out to pediatrics and neurology. It contains 
"semiqualitative" information about frequencies of various manifestations 
and how strongly one manifestation indicates one disease. Designed to guide 
physicians confronted with difficult diagnostic problems, CADUCEUS 
currently has a knowledge base of more than 600 diseases . 

"If you put this all together, you've got a knowledge base in internal 
medicine alone that is many times what the human brain could possibly 
remember," says Jack Myers, MD, prof essor-at-large (medicine) at the 
University of Pittsburgh. "I've built the thing, .and I can't remember the 
data ! " 

Myers, who constructed CADUCEUS with Harry E. Pople, PhD, associate 
professor of business administration at Pittsburgh, has been practicing 
internal medicine since the late 1930s. In the early 1970s, he began 
looking for a way to pass on his accumulated experience. "Rather than try 
to write another textbook, I figured it'd be better to try to put this into 
a computer where it could be manipulated and used," he says. He actually 
sought out computer scientists and interested them in his project. 

In keeping with its origins, CADUCEUS serves as an expert consultant 
offering a learned opinion that the physician "can take or leave, " says 
Myers. For example, a physician may complete a physical examination and 
find a certain kind of heart murmur. That kind of heart, murmur occurs in 
the descriptions of six diseases. The system draws these six out, along 
with "perfect descriptions" for all. 

Myers further explains that there are two numbers associated with 
heart murmur. The first one tells the likelihood of certain diseases given 
the heart murmur alone. The second number tells the likelihood of this 
heart murmur, given certain diseases. In reality, of course, there are many 
more pieces of information than "heart murmur, " but the basic process 
remains comparable: The computer comes up with the most likely diagnosis on 
the basis of these numbers. 

"The machine sets up a hypothesis for each disease and can rank its 
probability by comparing the known information to its ideal model, " says 
Myers. One big asset is flexibility. "In real life you rarely see a patient 
with every manifestation. And, like the human brain, CADUCEUS weighs 
probabilities between less-that-ideal descriptions." If the information 
given by the physician hasn't been sufficient for a conclusive diagnosis, 
CADUCEIS moves into an interrogative phase and asks for precise pieces of 
information . 

Why the Slow Growth of CMD? 

Although clinicians associated with the University of Pittsburgh often 



use CADUCEUS, both this system and ONCOCIN are far from routine use or 
widespread marketing. One problem is the incomplete knowledge base. "We 
cannot put the system out for routine use until those unknown diseases are 
fitted in in case the patient is unlucky enough to have one of those, " 

stresses Myers. "The next best fit isn't acceptable," 

Technology presents another problem. Although both Shortliffe and 
Myers are confident that these problems will disappear within the next 
decade, there is presently no small and affordable computer that can handle 
the typical AI program. 

Shortliffe goes even further in his reservations. "I don't think there 
is a becent decision-making system available to run on any computer that a 
physician would want to use on a regular basis," he says. Although not all 
workers in the field would agree, they can hardly deny that one or two 
routinely used systems for 20 years of research is not a good record. 

Technological limitations plague even investigators working on other 
than AI systems. "It's simply not true that computers can do everything," 
Ludwig emphasizes. "If you wanted to write a perfect program to play chess, 
it would have to look at every possible move and every possible sequence of 
moves at any given board position. That's something the fastest computers 
couldn't handle from now until hell freezes over." Since a perfect program 
is not possible, for either a chess program or a medical decision, choices 
of possibilities must be limited; how to do this is not at all clear. 

Poorly defined medical terminology and the uncertainty of medical 
knowledge itself are other bottlenecks for CMD systems, Ludwig says. 
Furthermore, notes Myers, computers still are not good at interpreting 
often crucial changes in patient attributes or laboratory values over time. 

But even without these limitations, the real difficulty has been 
physician acceptance, which has been withheld for much more than technical 
reasons . 

For one thing, if the decisions are useless, physicians won't rely on 
them — no matter how technically accurate they may be. According to 
Friedman, who has been working with CMD systems for more than a decade and 
believes strongly in their rltimate value, many of the programs make 
decisions that no one needs to make. 

Above all, he questions using computers for diagnostic decisions. 
"There are very few difficult decisions in medicine. The difficult 
decisions are the ones that don't fit the prior mold anyway, the ones that 
you don't have the data to base the decisions on in the first place." Most 
programs end up telling the physician what he knew to begin with. 

The really valuable system, contends Friedman, would decide what 
additional tests or questions would aid the physician's diagnosis and 
suggest them. "If we could use the computer to make good clinical 
decisions, then we could begin to talk about a diagnosis," he says. "The 
real questions are whether to do a test, whether it's cost effective, what 
is the real mortality, how effective is the therapy." 

One program that at least begins to answer these questions is the 
Present Illness Program (PIP) at Massachusetts Institute of Technology, 
Cambridge, and Tufts University, Medford, Mass. An AI program concerning 
edema, PIP simulates an expert taking a case history, thus suggesting 
important issues to less-expert physicians. "PIP won't outperform experts," 
acknowledges Stephen G. Pauker, MD, associate professor of medicine and 
chief of the position of clinical decision making at Tufts-New England 
Medical Center . "But most physicians aren ' t experts . " 

For a patient initially seen with massive pedal edema, PIP might ask: 
"Is pedal edema, which is massive, (Ref. lastig) (1) for days? (2) for 
weeks? (3) for months? (4) for years?" The physician responds and another 
multiple-choice question appears: "Is it (1) first time? (2) infrequent? 
(3) occasional? (4) frequent?" Depending on the responses , PIP might ask 
later about temporal patterns, symmetry, pain, and dyspnea, as well as 
about the patient ' s alcohol consumption, life insurance, and military 
history. 

The result is a literate paragraph summarizing the case and offering 
serveral diagnostic hypotheses (see figures, previous page) . 

Most researchers defend the usefulness of large knowledge bases in 
general, whether or not they suggest further tests or questions. As Marsden 
Blois, MD, professor of medical information science at the University of 
California, San Francisco, points out, "If the physician does not think of 
the correct diagnosis, he can't make it. A useful system can offer a menu 



or list of things that he might want to think about in face of a difficult 
diagnostic problem — like an uncommon disease that he may never have seen 
in his whole experience and may never have thought of since medical 
school." Blois' RECONSIDER serves this function by generating a 
differential diagnosis, based on the CMIT, given a list of patient 
findings . 

Usefulness can also depend on the demographic situation, "If you're in 
a tertiary care center where there are consultants all over the place, this 
kind of program may be less useful than in a rural area or medically 
underserviced area," says Shortliffe. 

Even if the programs are useful, however, physicians may be 
intimidated by unfamiliar computer languages. Most of the people building 
today's CMD systems take what they call the "user-interface" problem into 
account but, according to James A. Reggia, MD, PhD, most don't go far 
enough. Reggia, an assistant professor of neurology and mathematics at the 
University of Maryland, has designed what he calls a "knowledge Management 
System" (KMS) to obviate this problem. 

A KMS does not actually contain a knowledge base like CMD systems, but 
provides a framework into which a physician-author can insert his own 
knowledge base. It contains a collection of inference mechanisms (eg, 
Bayesian strategies or hypothesize-and-test models of diagnostic reasoning) 
that can operate on the new data. Because KMS uses a simple, "almost 
English" language, it allows a physician to implement a system specifically 
useful to him. 

"Some people think that our idea is crazy," says Reggia. "They think 
we'll never get physicians to be able to program computers. I don't believe 
so. " 

To prove his point, Reggia and colleagues gave about 4k medical 
students a course on the system and taught them how to build their own CMD 
systems. The students constructed a statistical pattern classification for 
thyroid diagnosis, for example, and a logical rule-based system to classify 
lung cancer. Regardless of past computer experience, students quickly and 
easily mastered the system. 

A less remediable problem may be what Shortliffe has called the almost 
"visceral" reaction to anything related to computers. But familiarity seems 
to be quickly wiping out most objections. Not only are computers becoming 
part of daily life, but costs continue to fall. Moreover, students coming 
into and finishing medical school frequently have computer experience, and 
many physicians have office computers that could easily accommodate CMD 
systems . 

Above all, as an increasing number of physicians begin to see 
computers as useful tools rather than threatening replacements, hostility 
is declining. "If a doctor wants to analyze the blood," explains de Dombal, 
"he doesn't go and sniff it himself: he uses a machine. And if he wants to 
know whether the bones are broken, he doesn't go feel them out for himself: 
he uses an X-ray machine. Now, for the first time, he can actually use a 
machine to help him with the symptoms as well; the computer decision is 
simply another piece of evidence." 

At the heart of the matter lies the question of whether a physician 
with this computer assistance generates significantly better patient care 
than a physician without it. For the most part, researchers have not yet 
answered this question. 

"Physicians are not stick-in-the-muds," stresses Friedman. "It's 
ridiculous to say they won't use computers. They use laser beams, they buy 
CT scanners for a million bucks apiece .... When these decision systems 
can do something that doctors can't do, every doctor will have one." 
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Computers can never replace physicians. Only physicians have the ability to 
reason logically and to mix that reason with intuition, experience, and 
ethics -- something no machine can ever learn to do." 

Or so many physicians have been telling people proposing 
computer-assisted medical decision making (CMD) since the early 1960s, and 
controversy has continued. But now it looks like the two sides are nearing 
agreement: Computers may never replace physicians, but they can and will 
assist their decisions. 

"Assist" is the key word in CMD. Early advocates tended to 
overestimate the powers of the computer and promised complete clinical 
evaluation at the push of a button. It was natural for physicians to feel 
somewhat threatened. It was also natural that, with experience, such 
promises had to be modified. 

Today there still are only several functioning knowledge-based 
decision systems — called "knowledge-based" because they make inferences 
using a data base of medical information stored in a computer's memory -- 
and they are extremely modest. More complex systems are currently under 
development, but even these will merely serve as expert consultants; the 
ultimate decision, which requires personal, occasionally subjective 
considerations, still will rest with the practitioner. 

"The early systems attempted to do diagnosis for the doctors which 
is crazy," says Francis T. de Dombal, MB, MD, consultant with the Leeds 
Area Health Authority and Reader in Clinical Information Science at the 
University of Leeds, England. "Doctors do some things very well indeed, and 
computers do them very badly. For example, if the Leeds United Football 
Team was playing at home, there 'd be quite a lot of people with bellyaches. 
The experienced doctor would know the source of all these bellyaches, and 
he wouldn't take everyone's appendix out. The computer might well do so." 

de Dombal is well acquainted with the abilities and limitations of 
computers: His Leeds system was one of the first and remains one of the few 
fully operative CMD systems. Able to discriminate between common causes of 
acute abdominal pain, the system has been used for the past eight years in 
three United Kingdom (UK) hospitals, and recently the UK government 
authorized and funded a three-year trial in 11 other hospitals. 

Although both the problem and the possible diagnoses are limited, the 
system has improved diagnostic accuracy for abdominal pain by 20%, 
according to de Dombal. After studying the pattern of admittances in his 
own hospital and others, he and his team constructed probabilities of 
common causes of abdominal pain and entered these into the computer memory 
along with the signs and symptoms associated with each cause. 

'What Have I Got, Doc?' 

When a patient is initially seen with abdominal pain, the examining 
physician records observations on a preprinted form that requests 
information about the pain, other symptoms, medical history, and findings 
from the general examination. The physician then gives the form to a member 
of the computing team, who enters it into the computer by choosing an 
"answer" for each of a series of categories. For example, under the 
category of "relieving factors," he can choose between "lying still," 
"vomiting," "antacids," "food," "other," and "nil." Other categories 
include sex, age, site of pain at onset, aggravating factors, and site of 
pain at present . 

Next the computer analyzes the data. Essentially, it compares the 



details of the new case with those of hundreds of similar cases in its 
memory and uses a theory of probabilities to calculate the likelihood of 
various diagnoses. The resulting printout displayed to the physician lists 
all the symptoms and gives the probabilities that . the patient has 
appendicitis, non-specific abdominal pain, diverticular disease, perforated 
peptic ulcer, cholecystitis, small bowel obstruction, or pancreatitis. A 
sample printout might suggest a 27.11% chance of appendicitis, a 12.28% 
chance of perforated peptic ulcer, and a 60.60% chance of nonspecific 
abdominal pain; the practitioner can combine this information with his 
clinical impressions to arrive at a diagnosis. 

"I would defend to the death the doctor's right to override what comes 
out of the computer if he thinks it's crazy," stresses de Dombal. "Right 
across the top of all our systems it says, 'Clinical judgment must .take 
precedence; these are only suggestions.'" 

After testing the system from January 1971 to August 1972, de Dombal 
found that preoperative diagnoses made with the computer-aided system 
(using data provided by the registrar) were 91% correct. This was in 
comparison with 42% correct for diagnoses made by unaided admitting 
physicians, 71% correct for diagnoses made by unaided house surgeons (less 
than six months experience) , 79% correct diagnoses by unaided registrars 
(between one and two years surgical experience) , and 81% correct diagnoses 
by unaided senior clinicians (more than five years surgical experience) . 
These results show one obvious advantage of the system: training 
inexperienced physicians. 

"What we're really trying to do," says de Dombal, "is to get the 
inexperienced doctor's level of performance up to that of the senior 
clinicians within a few weeks, as opposed to ten years." 

The improvement figures may be misleading, however, since the 
physician often makes the crucial decision before turning to the computer. 

"Before these trials were run, " says Richard Bruce Friedman, MD, 
associate professor of medicine and human oncology at the University of 
Wisconsin Medical School, Madison, "the investigators had already screened 
out every area but abdominal pain. By the time (Ref. aphysicia) knows it's 
abdominal pain, he calls the surgeon and he gets an x-ray even without a 
computer. The test is an artificial situation and makes no real difference 
to clinical procedure. 

"Even if you send this program neck pain, it will still make a 
diagnosis of abdominal pain." 

It seems to make a difference to some physicians, though. With the 
help of the World Organization of Gastroenterology, investigators have 
modified the data base to take geographical variation into account, as 
abdominal pain may have different causes in different areas. And since 
1972, the cost of the program has come down considerably: Once run on a 
large university computer, it now can run on small computers like Pets and 
^ples, which are quite common in hospitals and physicians' offices these 
days . 

Moreover, the US Navy is currently evaluating de Dombal 's program for 
use on submarines and is adapting the data base to this population. 
According to George Moeller, MD, director of the Behavioral Sciences Branch 
of the US Navy, appendicitis is one of the principal medical problems 
aboard submarines. Although some medical problems can be handled aboard, 
surgery of course requires moving the patient off the submarine. 

"That's something we would like to avoid," says Moeller, "not only 
because it (Ref. movigthepatiet ) would interrupt the mission but because 
it's a dangerous procedure in and of itself." Navy corpsmen could use the 
CMD program to effectively decide when to move the patient. Moeller adds 
that the Navy is also considering using a variation of the program for 
chest pain and major behavioral disorders. 

The only other general type of CMD system currently in use is almost 
the complete opposite of de Dombal ' s knowledge-based system: It covers a 
huge range of medical problems but uses no complicated statistical analyses 
to serve up probabilities of certain conditions or diseases. Such a system 
simply stores and retrieves huge bodies of text, allowing overbur-dened 
practitioners to deal with an "information explosion." 

Handling the 'Information Explosion' 

The National Library of Medicine, Bethesda, Md, for example, has 
developed a prototypical Hepatitis Knowledge Base that is in use. The 



American Medical Association and General Telephone and Electronics (GTE), 
moreover, will inaugurate the first nationwide electronic medical-health 
information system by late 1982. The program consists of a series of 
different data bases, some of which are closer to completion than others. 

The drug data base in particular seems almost ready, according to Dan 
Harris, director of the Medical Information Network at the AMA. To use this 
data base, the physician might type "Valium" into his office computer 
terminal (which can be hooked up to the GTE network) . He then would receive 
a kind of "menu" that would allow him to choose the kinds of information he 
would like, for example, on contraindications, pharmacokinetics, and drug 
interactions. Depending on the information requested, the computer will 
respond with information within seconds. 

A disease data base, taken largely from the AMA's Current Medical 
Information and Terminology (CMIT) , is under consideration now and would 
offer information on signs and symptoms, etiology, synonyms, and laboratory 
and pathology data for a named disease. 

"The computer can give this information more quickly than a reference 
book," contends Harris. "If you're a physician in an emergency room whose 
specialty happens to be surgery, and a patient walks in, you may first have 
to know what drugs he's involved with. If these drugs differ from those you 
normally prescribe, this system can give you what you need to know 
immediately. " 

Cost has been the traditional barrier to computer systems, but with 
increasingly common low-priced terminals, this barrier is on the way out. 
However, collecting a data base often takes years of time: Texts and 
journals have to be combed for information which then has to be 
reorganized; files have to be continually updated. For many experimental 
systems that combine huge data bases with complex programs and lofty 
aspirations, this lag time increases from years to decades. 

"It's easy to get the impression from reading (Ref. joural) papers 
that the day is around the corner when all doctors will be getting expert 
consultation from computers," says Dana Ludwig, MD, of the University of 
California, San Francisco. "That's not really true. Work has been going on 
for 20 years now, and there are very few systems being used in day-to-day 
practice anywhere — let alone in widespread use." 

On the other hand, because computers don't forget, overlook, tire, or 
equivocate, research is proceeding apace. There are plenty of incomplete 
but promising systems in experimental use, most often by physicians with 
computer science experience of their own. Very generally, these 
knowledge-based systems can be divided into two divergent types: 
numerically based and artificial intelligence (AI) systems. 

Experimental Systems 

Many of the numerically based systems resemble de Dombal's in that 
they use a statistical rule like Bayes theorem to make decisions. Bayes 
theorem provides a means for calculating the probability of hypothesis A, 
given the probability of hypothesis B. In medical terms, the theorem can be 
used to calculate the probability of each possible disease given a 
description of a particular case. 

One major problem of these Bayesian systems is that they must make 
assumptions that simply do not hold true in real medicine. Bayes theorem, 
as generally used, "doesn't conveniently allow for the hypothesis of 
multiple diseases in the same patient," says Ludwig, Moreover, "it doesn't 
conveniently allow the representation of nonindependence of symptoms within 
the disease . " 

In other words, most CMD systems using Bayes theorem define certain 
entities as patient attributes and other entities as disease hypotheses. 
They assume that the patient can have only one disease and that, assuming 
he has it, all the attributes — signs or symptoms -- occur independently 
of each other. Ludwig 's INFERNET system tries to avoid these obvious 
oversimplifications by applying an extension of Bayes theorem to a network 
of interconnected diseases, symptoms, and signs. 

"This system, " explains Ludwig, "allows the possibility of 
intermediate states. For example, the disease of interest might be 
myocardial infarction, and it might be causing shortness of breath and high 
pulmonary wedge pressure. But actually it causes shortness of breath and 
pressure through an intermediate stage — congestive heart failure. It's 
this intermediate state that links the two symptoms together." INFERNET can 



take this link into account to give a more flexible and realistic picture 
of probabilities. 

But INFERNET, like most other statistical-inference CMD systems, is 
nowhere near ready for clinical use. Not only must a vast amount of data be 
collected, but probabilities of specific diseases must be calculated — a 
time-consuming and expensive project. Beyond that, the questions remain: Do 
complex statistical calculations really model or surpass medical reasoning 
in terms of efficacy? Can the probabilities cranked out of the computer 
make any difference in patient care? 

Many physicians and computer specialists think not and have turned 
instead to that branch of computer science that attempts to simulate human 
reasoning: AI. Definitions of AI seem almost as plentiful as CMD systems, 
but generally AI can be defined as a type of program that generally 
manipulates ideas and symptoms rather than numbers. 

Artificial Intelligence Systems 

"The only way people would be willing to use (a typical statistical) 
program is if they were willing to turn over the decision to the computer, " 
says Edward H, Shortliffe, MD, PhD, assistant professor of medicine and 
computer science at Stanford University. "And I don't think doctors are 
likely to do that — ever — for a real patient .... If you run a 
program and it says your patient has systemic lupus, probability .89, then 
what do you do? You have no way of deciding if, in fact, the computer used 
the information properly or improperly. And although the computer may have 
asked very reasonable questions about signs and symptoms, how it came up 
with that number and whether or not you should believe it is another 
issue. " 

Shortliffe and other people working in AI believe that physicians 
should be able to evaluate the computer's reasoning and make conclusions as 
to the validity of its decision. Therefore, instead of building a complex, 
unintuitive formula into the machine, they try to model their systems on 
human cognition. The problem, of course, is that they first must learn how 
human cognition works. 

Part of this problem is being solved in the new field of medical 
decision making. Many CMD researchers — both in AI and other areas — are 
involved in the Society for Medical Decision Making and its international 
journal. One of the theories coming out of recent research, according to de 
Dombal (Nyhus LM: Computers and the surgeon. Surg Annual 1979; 2; 33-57), 
is that when physicians make decisions they acquire information, analyze 
it, and then decide the best course of management for the individual 
patient . 

These decisions can be either "algorithmic" as when a physician 
draws together many pieces of clinical data to arrive at a diagnosis — or 
"heuristic" — as when a physician asks a few questions, quickly thinks of 
a likely diagnosis, and switches to specific questions that will confirm or 
refute that diagnosis. A third type of decision — a more pragmatic one — 
is called "payoff": here the physician asks questions that will help him 
determine patient management. 

According to de Dombal, junior physicians and nonurgent problems 
usually evoke algorithmic decisions, while more senior physicians and 
urgent problems evoke heuristic and payoff approaches. Programs aiming to 
simulate the reasoning of an expert clinician, then, might follow a 
heuristic pathway. 

While building AI systems, researchers discover additional information 
about human cognition, which they add to later versions of their systems. 
Shortliffe worked essentially on a trial-and-error basis for his first 
system, MYCIN, as well as his current system, ONCOCIN.For example, he would 
run MYCIN on what he knew to be a meningitis case and see that the system 
performed poorly. He would then analyze its "reasoning" sequence to see 
where it went wrong and modify the program until it yielded a reasonable 
recommendation . 

According to Shortliffe, trial-and-error is "easier than asking an 
expert for everything he knows about meningitis and writing it down. It's 
very hard for people to analyze their own knowledge." One of ONCOCIN's 
major purposes is to extend basic knowledge about decision making and AI . 

Its other purpose is to develop a clinically useful oncology 
consultation system that can explain its reasoning to the physician. 
ONCOCIN was implemented in 1981 for experimental use in patients with 



lymphoma, and Shortliffe hopes to expand the program to therapies of many 
other cancers. By storing lengthy and complex protocols in the computer 
memory, he hopes to use ONCCX:iN to greatly improve patient care and 
eliminate departures form protocol guidelines. 

Like its predecessor MYCIN, ONCOCIN bases its decisions on a series of 
"If-then" rules. Depending on answers to the "if" condition, provided by 
the physician, the computer uses its preprogrammed "then" response as the 
"if" condition for the next rule. An English "translation" of the rule for 
determining "attenuated dose" is: 

RULE 075 

To determine the current attenuated dose for all drugs' in MOPP 
(nitrogen mustard, Oncovin, procarbazine, prednisone) or for all drugs in 
PAVE (procarbazine, Adriamycin, and Velban) 

If: (1) This is the start of the first cycle after a cycle was 
aborted, and 

(2) The blood counts do not warrant dose attenuation 

Then: Conclude that the current dose is 75% of the previous dose. 

ONCOCIN also draws on stored data files containing attributes of other 
patients as well as rules about drug usage, tests, and other information 
relevant for protocol decision making. Combining this entire knowledge base 
with information about the new patient's diagnosis and test results, 
ONCOCIN produces a recommendation, along with an explanation of its 
decision, about five minutes after the physician starts entering data. 

Another experimental system using AI, but for a much larger area of 
medicine, is CADUCEUS (formerly INTERNIST) at the University of Pittsburgh. 
The program, supported by the National Institutes of Health and the 
National Library of Medicine, initially was limited to internal medicine 
but is currently branching out to pediatrics and neurology. It contains 
"semiqualitative" information about frequencies of various manifestations 
and how strongly one manifestation indicates one disease. Designed to guide 
physicians confronted with difficult diagnostic problems, CADUCEUS 
currently has a knowledge base of more than 600 diseases . 

"If you put this all together, you've got a knowledge base in internal 
medicine alone that is many times what the human brain could possibly 
remember," says Jack Myers, MD, prof essor-at-large (medicine) at the 
University of Pittsburgh. "I've built the thing, and I can't remember the 
data!" 

Myers, who constructed CADUCEUS with Harry E. Pople, PhD, associate 
professor of business administration at Pittsburgh, has been practicing 
internal medicine since the late 1930s. In the early 1970s, he began 
looking for a way to pass on his accumulated experience. "Rather than try 
to write another textbook, I figured it'd be better to try to put this into 
a computer where it could be manipulated and used," he says. He actually 
sought out computer scientists and interested them in his project. 

In keeping with its origins, CADUCEUS serves as an expert consultant 
offering a learned opinion that the physician "can take or leave," says 
Myers. For example, a physician may complete a physical examination and 
find a certain kind of heart murmur. That kind of heart murmur occurs in 
the descriptions of six diseases. The system draws these six out, along 
with "perfect descriptions" for all. 

Myers further explains that there are two numbers associated with 
heart murmur. The first one tells the likelihood of certain diseases given 
the heart murmur alone. The second number tells the likelihood of this 
heart murmur, given certain diseases. In reality, of course, there are many 
more pieces of information than "heart murmur," but the basic process 
remains comparable: The computer comes up with the most likely diagnosis on 
the basis of these numbers. 

"The machine sets up a hypothesis for each disease and can rank its 
probability by comparing the known information to its ideal model," says 
Myers. One big asset is flexibility. "In real life you rarely see a patient 
with every manifestation. And, like the human brain, CADUCEUS weighs 
probabilities between less-that-ideal descriptions." If the information 
given by the physician hasn't been sufficient for a conclusive diagnosis, 
CADUCEIS moves into an interrogative phase and asks for precise pieces of 
information. 

Why the Slow Growth of CMD? 

Although clinicians associated with the University of Pittsburgh often 



use CADUCEUS, both this system and ONCOCIN are far from routine use or 
widespread marketing. One problem is the incomplete knowledge base. "We 
cannot put the system out for routine use until those unknown diseases are 
fitted in — in case the patient is unlucky enough to have one of those, " 
stresses Myers. "The next best fit isn't acceptable," 

Technology presents another problem. Although both Shortliffe and 
Myers are confident that these problems will disappear within the next 
decade, there is presently no small and affordable computer that can handle 
the typical AI program. 

Shortliffe goes even further in his reservations. "I don't think there 
is a becent decision-making system available to run on any computer that a 
physician would want to use on a regular basis," he says. Although not all 
workers in the field would agree, they can hardly deny that one or two 
routinely used systems for 20 years of research is not a good record. 

Technological limitations plague even investigators working on other 
than AI systems. "It's simply not true that computers can do everything," 
Ludwig emphasizes. "If you wanted to write a perfect program to play chess, 
it would have to look at every possible move and every possible sequence of 
moves at any given board position. That's something the fastest computers 
couldn't handle from now until hell freezes over." Since a perfect program 
is not possible, for either a chess program or a medical decision, choices 
of possibilities must be limited; how to do this is not at all clear. 

Poorly defined medical terminology and the uncertainty of medical 
knowledge itself are other bottlenecks for CMD systems, Ludwig says. 
Furthermore, notes Myers, computers still are not good at interpreting 
often crucial changes in patient attributes or laboratory values over time. 

But even without these limitations, the real difficulty has been 
physician acceptance, which has been withheld for much more than technical 
reasons . 

For one thing, if the decisions are useless, physicians won't rely on 
them — no matter how technically accurate they may be. According to 
Friedman, who has been working with CMD systems for more than a decade and 
believes strongly in their rltimate value, many of the programs make 
decisions that no one needs to make. 

Above all, he questions using computers for diagnostic decisions. 
"There are very few difficult decisions in medicine. The difficult 
decisions are the ones that don't fit the prior mold anyway, the ones that 
you don't have the data to base the decisions on in the first place." Most 
programs end up telling the physician what he knew to begin with. 

The really valuable system, contends Friedman, would decide what 
additional tests or questions would aid the physician's diagnosis and 
suggest them. "If we could use the computer to make good clinical 
decisions, then we could begin to talk about a diagnosis," he says. "The 
real questions are whether to do a test, whether it's cost effective, what 
is the real mortality, how effective is the therapy." 

One program that at least begins to answer these questions is the 
Present Illness Program (PIP) at Massachusetts Institute of Technology, 
Cambridge, and Tufts University, Medford, Mass. An AI program concerning 
edema, PIP simulates an expert taking a case history, thus suggesting 
important issues to less-expert physicians. "PIP won't outperform experts," 
acknowledges Stephen G. Pauker, MD, associate professor of medicine and 
chief of the position of clinical decision making at Tufts-New England 
Medical Center. "But most physicians aren't experts." 

For a patient initially seen with massive pedal edema, PIP might ask: 
"Is pedal edema, which is massive, (Ref. lastig) (1) for days? (2) for 
weeks? (3) for months? (4) for years?" The physician responds and another 
multiple-choice question appears: "Is it (1) first time? (2) infrequent? 
(3) occasional? (4) frequent?" Depending on the responses, PIP might ask 
later about temporal patterns, symmetry, pain, and dyspnea, as well as 
about the patient's alcohol consumption, life insurance, and military 
history. 

The result is a literate paragraph summarizing the case and offering 
serveral diagnostic hypotheses (see figures, previous page) . 

Most researchers defend the usefulness of large knowledge bases in 
general, whether or not they suggest further tests or questions. As Marsden 
Blois, MD, professor of medical information science at the University of 
California, San Francisco, points out, "If the physician does not think of 
the correct diagnosis, he can't make it. A useful system can offer a menu 



or list of things thit he might want to think about in face of a difficult 
diagnostic problem — like an uncommon disease that he may never have seen 
in his whole experience and may never have thought of since medical 
school. " Blois' RECONSIDER serves this function by generating a 
differential diagnosis, based on the CMIT, given a list of patient 
findings . 

Usefulness can also depend on the demographic situation. "If you're in 
a tertiary care center where there are consultants all over the place, this 
kind of program may be less useful than in a rural area or medically 
underserviced area," says Shortliffe. 

Even if the programs are useful, however, physicians may be 
intimidated by unfamiliar computer languages. Most of the people building 
today's CMD systems take what they call the "user-interface" problem into 
account but, according to James A. Reggia, MD, PhD, most don't go far 
enough. Reggia, an assistant professor of neurology and mathematics at the 
University of Maryland, has designed what he calls a "knowledge Management 
System" (KMS) to obviate this problem. 

A KMS does not actually contain a knowledge base like CMD systems, but 
provides a framework into which a physician-author can insert his own 
knowledge base. It contains a collection of inference mechanisms (eg,- 
Bayesian strategies or hypothesize-and-test models of diagnostic reasoning) 
that can operate on the new data. Because KMS uses a simple, "almost 
English" language, it allows a physician to implement a system specifically 
useful to him. 

"Some people think that our idea is crazy," says Reggia. "They think 
we'll never get physicians to be able to program computers. I don't believe 
so. " 

To prove his point, Reggia and colleagues gave about 4k medical 
students a course on the system and taught them how to build their own CMD 
systems. The students constructed a statistical pattern classification for 
thyroid diagnosis, for example, and a logical rule-based system to classify 
lung cancer. Regardless of past computer experience, students quickly and 
easily mastered the system. 

A less remediable problem may be what Shortliffe has called the almost 
"visceral" reaction to anything related to computers. But familiarity seems 
to be quickly wiping out most objections. Not only are computers becoming 
part of daily life, but costs continue to fall. Moreover, students coming 
into and finishing medical school frequently have computer experience, and 
many physicians have office computers that could easily accommodate CMD 
systems . 

Above all, as an increasing number of physicians begin to see 
computers as useful tools rather than threatening replacements, hostility 
is declining. "If a doctor wants to analyze the blood," explains de Dombal, 
"he doesn't go and sniff it himself: he uses a machine. And if he wants to 
know whether the bones are broken, he doesn't go feel them out for himself: 
he uses an X-ray machine. Now, for the first time, he can actually use a 
machine to help him with the symptoms as well; the computer decision is 
simply another piece of evidence." 

At the heart of the matter lies the question of whether a physician 
with this computer assistance generates significantly better patient care 
than a physician without it. For the most part, researchers have not yet 
answered this question. 

"Physicians are not stick-in-the-muds, " stresses Friedman. "It's 
ridiculous to say they won't use computers. They use laser beams, they buy 
CT scanners for a million bucks apiece .... When these decision systems 
can do something that doctors can't do, every doctor will have one." 
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ABSTRACT: DXplain is an evolving computer-based diagnostic 
decision-support system designed for use by the physician who has no 
computer expertise. DXplain accepts a list of clinical manifestations and 
then proposes diagnostic hypotheses. The program explains and justifies its 
interpretations and provides access to a knowledge base concerning the 
differential diagnosis of the signs and symptoms. DXplain was developed 
with the support and cooperation of the American Medical Association. The 
system is distributed to the medical community through AMA/NET — a 
nationwide computer communications network sponsored by the American 
Medical Association — and through the Massachusetts General Hospital 
Continuing Education Network. A key element in the distribution of DXplain 
is the planned collaboration with its physician-users whose comments, 
criticisms, and suggestions will play an important role in modifying and 
enhancing the knowledge base, 

SHORTLIFFE (Ref. 1) defines a computer-based medical decision-support 
system as a computer program designed to help health professionals make 
clinical decisions. Under this broad definition, there are many 
decision-support systems in use today. They include such applications as 
clinical laboratory systems, pharmacy systems, and radiology reporting 
systems. For the most part, these are "passive"- information management 
tools. That is, they make information more readily available to the 
physician but they do not provide "active" decision support by applying 
medical knowledge to a specific patient's data, nor do they recommend a 
specific conclusion or course of action. 

There has been less success in developing active systems. One strategy 
used in such systems applies predefined rules to the patient's data and 
alerts providers to conditions that might require action. The most 
successful system for inpatient care is the HELP system. (Ref. 2) In 
ambulatory medicine, two examples of systems that provide patient-specific 
recommendations for ambulatory care are CARE (Ref. 3) and COSTAR. (Ref. 4) 
There are also a few examples of "expert" systems that guide the physician 
in treating certain complex, but well-defined, conditions, eg, the ONCOCIN 
system. (Ref. 5) These systems contain in-depth knowledge about a 
well-defined subset of medical knowledge and require considerable and 
detailed information about the patient's clinical status to provide the 
computer-generated consultations . 

Computer-aided diagnosis in general medicine requires a much broader 
level of decision support. There have been a number of provocative research 
efforts in computer-aided diagnosis, (Ref. 6-9) but these have been 
research prototypes whose contributions were primarily methodological. With 
the exception of the work by deDombal et al, (Ref. 10) these programs were 
not made available to the practicing physician. There are a number of 
reasons to explain the limited impact of information technology in the 
diagnostic aspects of medical decision making. (Ref. 1,11,12) Major 




problems have included difficulty in accessing and interacting with the 
systems, a limited area of application (eg, one of the most successful 
computer-aided diagnosis programs is limited to the differential diagnosis 
of the acute abdomen (Ref. 10)), a limited ability of the systems to 
explain and justify the interpretations, and less than acceptable quality 
of the interpretations. Some investigators believe that there are major 
intellectual and technical problems that must be solved before there can be 
a truly reliable consulting program. In particular, it is claimed that 
computer programs must be able to incorporate pathophysiological knowledge 
and causal relationship based on pathophysiological reasoning. (Ref. 12) 

We accept the validity of these concerns, but take a more optimistic 
position concerning the potential for using currently available knowledge 
and technology to assist the physician in the information needs of daily 
practice. A recent study suggests that the information needs of physicians 
in office practice are not being met by printed sources such as textbooks 
and journal articles. (Ref. 13) Of the practice-related questions that were 
identified, 25% involved the diagnosis of symptoms, physical findings, or 
syndromes. We believe that a significant portion of the practical needs of 
clinical medicine can be met now by providing ready access to a 
computer-stored knowledge base of diagnoses and their associated signs and 
symptoms . 

We believe it is possible to develop a computer program that uses an 
extensive knowledge base and relatively simple computational models to 
provide significant diagnostic problem-solving assistance to the practicing 
physician. The program should allow the computer-naive user to enter a set 
of signs and symptoms using a typical medical vocabulary. The program would 
then generate a list of hypotheses that deserve consideration, comment on 
the diagnostic relevance of each sign and symptom, and suggest specific 
additional data elements that might clarify the differential diagnosis 
currently favored by the computer model. (Ref. 9,14-16) 

This article describes DXplain, an evolving computer-based diagnostic 
decision-support system designed for use by the physician who has no 
computer expertise. (Ref. 14) DXplain accepts a list of clinical 
manifestations and then proposes diagnostic hypotheses. DXplain explains 
and justifies its interpretations and provides easy access to a 
comprehensive knowledge base concerning the differential diagnosis of the 
set of signs and symptoms. 

DXplain does not attempt to make a single diagnosis to mimic the 
behavior or replace the judgment of the expert clinician. DXplain has a 
less ambitious, but perhaps more attainable goal: to suggest a list of 
diagnoses that should be considered given a particular set of signs and 
symptoms. Because DXplain is in a continuing state of development, we have 
labeled it an evolving diagnostic decision-support system. Our plan is that 
DXplain will continue to improve through collaboration with its 
physician-users whose comments, criticisms, and suggestions will play an 
important role in modifying and enhancing the knowledge base and the 
algorithms used in the computer-generated interpretations. 

SYSTEM CAPABILITIES 

We believe that the following six criteria are important for a • 
differential diagnosis decision-support system intended for use in routine 
clinical practice. The system should (1) be easy to use by physicians who 
have little or no computer background; (2) be based on comprehensive 
medical content; (3) provide correct and accurate interpretations; (4) 
justify its interpretations; (5) be convenient to access from the 
physician's office, hospital, or home; and (6) evolve and improve as a 
result of user criticism and analysis of user sessions. 

Easy to Use 

It has been the general experience that few practicing physicians will 
use computer programs requiring extensive training or knowledge of computer 
technology. DXplain demands little knowledge of computer technology and 
requires only the use of the vocabulary common in medical practice. On-line 
help is available for an explanation of the system commands. 

As with any decision-support system, DXplain uses a controlled 
vocabulary for communicating patient information to the system. Computer 
technology has not yet progressed to the stage in which a computer program 
can recognize the free-form narrative text that a physician might use to 



describe clinical manifestations in a medical record or with a colleague. 
The predefined medical vocabulary used by DXplain consists of more than 
4700 terms based on the clinical and basic laboratory data that might be 
collected in an ambulatory practice or in the emergency department. 

DXplain has several features that assist the user in selecting the 
desired terms from the controlled vocabulary. For instance, there is 
extensive synonym and abbreviation terminology; also, the system can 
recognize and correct many misspellings. DXplain can recognize terms that 
are "close" (eg, "congestive heart failure" and "heart failure"), as well 
as synonyms for terms at both the single-word level (eg, "kidney" and 
"renal" are equivalent in many different terms) and at the fullphrase level 
(eg, "anisocoria" is equivalent to "pupillary inequality," "factor VIII 
deficiency" is considered equivalent to "hemophilia"; and "blood glucose 
elevated" is considered equivalent to "hyperglycemia"). In addition, many 
abbreviations are recognized (eg, "ESR" and "CHF") to speed data entry. 

The DXplain vocabulary structure is hierarchical, which allows 
flexibility in specifying the precise level of detail of the clinical 
manifestation. Thus, DXplain groups more specific terms (such as "lower 
abdominal pain") under a less specific term (such as "abdominal pain"). 
This hierarchy is important for both the user interface and the algorithm 
used for interpretation. When a user enters a specific term, such as "right 
lower quadrant pain," all of the appropriate less-specific terms, such as 
"lower abdominal pain" and "abdominal pain, " are assumed by the system, but 
related terms at the same level of specificity, such as "left lower 
quadrant pain," are not assumed. 

A typical user interaction with DXplain is illustrated in Fig 1. 
Examples of how the system helps the user select the appropriate terms 
during input are illustrated in Fig 2. 

A second important factor relating to ease of use is the rapidity with 
which the user can enter the clinical manifestations and extract the 
desired information and interpretation. The system is designed to be 
largely self-explanatory. There is no need to read manuals: the beginning 
user can take advantage of menu selection; the experienced user can make 
use of an abbreviated command language to speed the interaction. The 
requirement for rapid time response was a critical design factor that 
influenced decisions made about the organization of the knowledge base in 
the computer. It requires about two minutes to complete the dial-in 
sequence to log on to AMA/NET and to connect to the computer located at 
Massachusetts General Hospital. The entry of the seven terms illustrated in 
Fig 1 required about one minute. Evaluation of the clinical data by the 
computer program and presentation of the interpretation and list of 
diagnostic hypotheses takes from 10 to 20 s. 

Accurate and Comprehensive Medical Content 

The development of a comprehensive knowledge base in medicine, whether 
printed or computer based, is a large effort. The merit of any 
decision-support system depends to a significant extent on the quality of 
the knowledge base used in formulating its recommendations. One aspect of 
any knowledge base, whether printed or computer based, is the inability of 
the author to certify that the entire knowledge base is totally accurate 
and comprehensive. 

There are no methods to extract automatically the relevant information 
from the published literature. In fact, the published literature is not as 
useful as desired since it often does not provide the quantitative 
relatrionships between clinical manifestations and diseases that are 
required for a decision-support system. For example, one frequently finds 
statements like "symptom X is 'occasionally found' in the disease" or "it 
is 'not uncommon' to have symptom Y" or "the 'great majority' of the 
patients with the disease will demonstrate symptom Z." 

To provide the necessary knowledge base for DXplain, we begin with the 
computer-based version of Current Medical Information and Terminology 
(CMIT), which is published and supported by the American Medical 
Association. (Ref. 17) Current Medical Information and Terminology contains 
summaries of information concerning the etiology, signs and symptoms, 
laboratory findings, and disease course for more than 3000 distinct 
diseases, disorders, and conditions. 

To transform this knowledge base into one suitable for a 
decision-support system, it was necessary to carry out a considerable 



amount of manipulation and additional content acquisition. For the disease 
list for DXplain, we combined some diagnoses from CMIT to form more 
inclusive definitions, while others were subdivided into more specific 
forms of the disease (eg, separate stages of a disease, forms of the 
disease specific to certain demographic groups, and forms attributed to 
different etiologic agents) . Additional diseases were added when 
deficiencies in CMIT were noted. 

As the first step in specifying the controlled vocabulary of terms to 
represent the clinical features of each disease, terms were extracted from 
the text of CMIT, yielding a list of some 6000 words and phrases. An 
extensive review of this list was undertaken to lend coherence to the 
vocabulary. Synonyms were merged where appropriate and similar terms were 
linked together in a hierarchical manner to express their relationships 
(usually in terms of such qualifiers as severity, duration, or anatomic 
location). The outcome was a term directory consisting of 4000 descriptors. 
As disease descriptions were compiled, 700 additional concepts were added 
to the directory. 

We used a three-step process to generate the disease-term 
relationships of the knowledge base. First, a list was compiled from the 
CMIT data base and from expert judgment of all the terms that had some role 
in either supporting or ruling out a particular disease. Next, medical 
texts were reviewed to determine the frequency of each clinical finding in 
the disease. Where necessary, in circumstances of recent discoveries or 
obscure conditions, literature searches were conducted to supplement 
standard references. Finally, an estimate of the potential for the presence 
or absence of each term to evoke or refute the diagnosis was made. 

This content development was an iterative process that was done by the 
system developers in collaboration with 13 physicians representing several 
medical disciplines. Because of the number of individuals involved, 
extensive discussions were required to reach a consensus on content and 
assignment of appropriate weighting factors for the relationship of terms 
to diseases. Once guidelines were established, authors were trained in 
filling out work sheets for each disease. Each work sheet included all 
terms that had been previously assigned, a prescribed set of demographic 
terms (age, sex, and duration of symptoms), and a generous number of blank 
lines for adding new terms. After the work sheets were completed, one of us 
(J.J.C.) reviewed them to ensure consistency of content, terminology, and 
assignment of evoking potentials. 

The result of this work is DXplain *s knowledge base consisting of 
descriptions of approximately 2000 diseases, about 4700 terms (signs, 
symptoms, and more) , and some 65 000 relationships among them. On average, 
each disease description contains relationships to 35 terms. Each 
relationship identifies the frequency of the manifestation among patients 
with the disease and the strength with which a manifestation evokes a 
diagnosis for consideration. 

From a practical viewpoint, it is impossible to verify completely all 
the elements in a knowledge base that is the size and complexity of 
DXplain. We believe that there must be a continuing effort to detect and 
correct incomplete or inaccurate disease descriptions. In addition, the 
knowledge base can never be static, but must continue to evolve as new 
knowledge is gained and new evidence is identified about each disease. 

One of the major advantages of having the knowledge base reside on a 
single central computer is that the knowledge base can be updated easily 
and as often as necessary. The addition of a new term, disease, or synonym, 
the modification of the descriptor-disease relationship, or the enhancement 
of the program can be done quickly. Because these changes are immediately 
available to all users, the system can be dynamically responsive to user 
suggestions concerning deficiencies and improvements. 

Provide Correct and Accurate Interpretations 

In designing a differential diagnosis decision-support system, we 
assumed that the most useful assistance that can be provided to the 
practicing physician is to suggested the reasonable diagnoses that should 
be considered, given any particular set of signs and symptoms. We believe 
that in many complex diagnostic problems (and many not so complex), a major 
reason for not making the correct diagnosis is the failure of the physician 
to generate a differential diagnosis list that is comprehensive enough to 
include the correct diagnosis, yet sufficiently focused to include only the 



more likely diagnostic possibilities. 

The goal in DXplain is to assist the physician in two phases of the 
process of differential diagnosis: (1) to remind the physician of diseases 
that should be considered as possible candidates to explain the patient's 
condition and (2) to provide information about specific diseases that might 
be eliminated from consideration. We explicitly reject the objective of 
determining the "correct" diagnosis. We reason that the clinician will 
always have a more complete picture of the patient than will be entered 
into the computer program and that a more realistic goal for a 
decision-support system is to bring to the user's attention the plausible 
explanations for a patient's signs and symptoms. The user may then consider 
which diseases are appropriate to the case by applying common sense, 
clinical experience, and full knowledge about the patient and, possibly, by 
further interacting with the program. 

DXplain is not intended to aid the specialist working in his/her 
specialty area. Thus, the cardiologist dealing with a patient with a 
complex murmur is unlikely to find DXplain very helpful; but the 
cardiologist dealing with a patient with abdominal pain may obtain useful 
assistance. The most obvious way in which DXplain can help is in suggesting 
obscure or rare diseases that may be rarely seen by most physicians. 
DXplain also may be helpful by suggesting diseases that present in atypical 
ways . 

DXplain uses a relatively straightforward algorithm to select its list 
of plausible diagnoses. This algorithm has been described in a previous 
publication. (Ref . 14 ) The DXplain selection rule takes advantage of 
conditional probabilities and a scoring system similar to a Bayesian 
computation. Models such as this have been frequently used in other 
diagnostic decision-support projects, the most influential being the 
ranking algorithm used in INTERNIST/QMR. (Ref. 7,14) Our development of 
DXplain was strongly influenced by the experiences and the limitations of 
the original efforts of Miller and colleagues with INTERNIST, and their 
later development of QMR. 

The knowledge base for DXplain contains more than 65 000 relationships 
between diseases and patient descriptors {or clinical manifestations) . The 
format for these relationships and the data structures are similar to the 
format and data structures used in the INTERNIST/QMR system (Ref. 7) except 
that in DXplain terms cannot only support a given diagnosis, but can also 
contradict other diagnoses. The relationships in DXplain are concerned with 
three different elements: (1) term importance, (2) term frequency, and (3) 
term-evoking power. 

Term importance is used to express how significant the particular term 
is in indicating the presence of disease. A high term importance is given 
to findings that can be identified with high reliability or are rarely 
found in healthy persons and, therefore, should be explained by some 
disease within the differential diagnoses. 

Term frequency is used to identify how often a particular term is 
expected to occur in a specific disease. The possible values for term 
frequency can be one of seven different states ranging from "always" to 
"never . " 

Term-evoking power is used to identify how strongly a particular term 
supports the possibility that a specific disease might be present. This 
term is related to the concept of predictive value positive (the predictive 
value of a positive test — this is the probability of a disease being 
present given the presence of a certain finding) . The possible values for 
term-evoking power can be one of eight different states ranging from 
"certainly supported" to "weakly supported" and from "weakly contradicted" 
to "strongly contradicted." 

DXplain uses the numerical values of these different relationships to 
derive the list of the diagnoses that should be considered (Fig 1) . DXplain 
selects diseases to be included on this list on the basis of how well the 
manifestations of a particular disease match the descriptors that have been 
entered by the user. The diseases are presented to the user in two lists: 
"common diseases" and "rare diseases"; in addition, a serious disease (one 
that may require relatively immediate action) is so indicated on the list 
by an asterisk. 

The user can request DXplain to explain why any specific diagnosis was 
included (Fig 3) . DXplain will present the clinical findings entered by the 
user that support the selection of that disease, the clinical findings that 



would not be expected in that disease, and additional clinical findings 
that would be expected if that disease was present. In this way, DXplain 
assists the user in understanding the logic used by the program and 
facilitates pattern matching by the user in comparing DXplain 's disease 
description with his/her knowledge about the patient. The intent is to 
present sufficient information so that the user can always use has/her own 
clinical judgment as to the appropriateness of DXplain' s interpretation. 

The user can ask DXplain to consider a specific diagnosis that was not 
included on the initial list (Fig 3). DXplain will then present the same 
analysis, as described in the previous paragraph, for the diagnosis under 
consideration and, furthermore, will include this diagnosis in any later 
interpretation after additional findings are entered. 

The user can change DXplain into an interrogative mode wherein the 
system will question the user about the presence or absence of significant 
findings that have the potential for clarifying DXplain 's current 
differential diagnoses (Fig 4), This mode is particularly useful in helping 
the user select the clinical manifestations that are important without 
forcing them to enter a large number of less relevant findings. At any time 
in this mode, the user can interrupt DXplain to ask "Why?" ie, to ask 
DXplain to justify why this particular clinical manifestation is important. 
DXplain will respond by displaying the name of the disease that is being 
considered at that point in the interaction and the reason the particular 
finding might be important in conforming the presence of that disease, 

DXplain 's ability to explain and justify are key elements of the 
system. It is critical that this system not be perceived as a magic black 
box that can somehow provide the "answer" to a complex diagnostic problem. 
We believe that physicians will not accept DXplain as a useful diagnostic 
assistant unless the clinical interpretations seem reasonable and unless 
the system can offer explanations that are understandable and persuasive 
(Ref. 18) 

Convenient to Access 

Almost a decade ago, Shortiiffe wrote: "A recurring observation as one 
reviews the literature of computer-based medical decision making is that 
essentially none of the systems has been effectively utilized outside of a 
research environment, even when its performance has been shown to be 
excellent." (Ref. 8) For the most part, this observation is still true 
today. One important barrier is that the practicing physician cannot easily 
access a computer-based decision-support system from his/her office, or any 
other location, at any time. There are a number of systems that are 
available in university hospitals and that are of great importance in the 
host institution and have considerable value as demonstration models. 
However, none of these systems provide support to the practicing physician 
on a national scale. 

DXplain is unique in that the decision-support capability is easily 
accessible using only a simple computer terminal (a microcomputer can also 
be used to make the connection), a telephone modem, and a telephone call 
(usually a local number in most of the major cities of the United States) . 
DXplain is also available in a similar fashion in Canada and Japan. There 
is no start-up cost associated with purchase of the programs or of the 
knowledge base. Many physicians already have the necessary technology to 
access DXplain since the needs are the same as those used for on-line 
bibliographic search services. 

DXplain was developed with the support of the American Medical 
Association and is designed to be distributed to the medical community 
through AMA/NET — a nationwide computer communications network sponsored 
by the American Medical Association. Physicians and other professionals can 
access the system through the AMA/NET . Medical schools and teaching 
hospitals can access the system either through AMA/NET or through the 
Massachusetts General Hospital Continuing Education Network. In both cases, 
the cost of accessing DXplain is directly dependent on the length of time 
one is connected to the system. 

A subscriber to AMA/NET can also access information data bases 
(EMPIRES clinical reference citations. Medical Procedure Coding and 
Nomenclature, the Associated Press Medical News Service, and more), public 
information services (Centers for Disease Control Information Service, 
National Library of Medicine/National Institutes of Health Information 
Services, Adverse Drug Reaction Reporting Form, and more), electronic 



mail, and the Massachusetts General Hospital interactive medical education 
courses (Hoffer et al (Ref. 19)). AMA/NET also provides documentation and 
telephone support to its subscribers. (For information on AMA/NET, call 
1-800-426-2873; for information on the Massachusetts General Hospital 
Continuing Education Network, call 617-726-3950.) 

Evolve and Improve as a Result of User Criticism . 

A key element in the distribution of DXplain is an integrated 
electronic mail capability. At any point in the interaction a user may 
enter a comment or question into the computer system. This electronic mail 
is read at frequent intervals by the system developers at Massachusetts 
General Hospital and responded to as appropriate. 

We view the plan for the continuing improvement and enhancement of 
DXplain as one of the more important aspects of its development. The major 
potential weakness of any diagnostic decision-support system such as 
DXplain is the quality and completeness of the underlying knowledge base. 
Evaluating a clinical decision-support system is difficult, both 
conceptually and in practice. {Ref. 1,11) Systematic clinical trials or 
formal outcome studies on the impact either of computer-based knowledge 
bases or of medical textbooks are logistically almost impossible. DXplain 
has been used by physicians for over 500 hours at more than 40 different 
test sites in the United States, Canada, and Japan. The initial user 
acceptance and peer review has been favorable, although the evaluation has 
been largely anecdotal. 

The continuing refinement of DXplain will be most fruitful if the 
planned collaboration between the physician-users and the developers 
materializes. We expect and need the participation of physicians who will 
challenge the system with rare diseases and with uncommon manifestations of 
common diseases. The continuing critical review by DXplain users of the 
knowledge base and interpretations of the system will provide important 
feedback in the iterative process of knowledge base development. 

SYSTEM LIMITATIONS 

DXplain has a knowledge base that covers more diseases than are 
discussed in most textbooks of medicine , but in some areas DXplain is 
incomplete, eg, there is only limited coverage of dermatologic diseases , 
where diagnosis often depends on the visual appearance of the lesion. At 
present, there is only minimal coverage of diseases from psychiatry and 
orthopedics. DXplain presently allows the entry of only a limited set of 
laboratory test findings. The original design goal of DXplain was primarily 
focused on ambulatory medicine ; as a result, the current version of 
DXplain does not allow the entry of many of the complex laboratory tests 
that are performed only in hospitals. We are continuing to add both 
diseases and terms, including the most common laboratory abnormalities. 

DXplain can cope in only a limited fashion with the variations in the 
way that a disease can present based on its evolution over time, degree of 
severity, and the modifications introduced by therapy. In addition, DXplain 
does not identify complex disease patterns caused by the presence of two or 
more diseases in the same patient. DXplain considers each disease and its 
expected manifestations as unique entities. DXplain is unable to recognize 
how the manifestations of one disease can be modified by the presence of a 
second interacting disease. However, DXplain will attempt to select and 
present to the user all the individual diseases that might account for the 
more important findings so that the physician can use clinical judgment to 
carry out any appropriate recognition of disease patterns. 

A number of authors have emphasized the importance of an explanation 
capability to encourage physicians to use decision-support systems. (Ref. 
1,18) DXplain has an explanatory capability, but it is limited to the 
justification of why a particular disease should be considered (or ruled 
out), based solely on the likelihood of occurrence of the specific clinical 
manifestations in that disease . DXplain has no pathophysiological or 
anatomic knowledge and no ability to consider pathophysiological or 
anatomic reasoning. 

The mere existence of data in a knowledge bank is, of itself, no 
guarantee of completeness and accuracy. The same professional judgment and 
critical appraisal are required when using DXplain as required when reading 
a medical textbook or discussing a patient case. In fact, since the 
computer program does not possess the depth of medical knowledge, the 



wisdom of medical experience, or the ability of a colleague to reason, it 
would be wise to be even more critical of the computer's interpretation. 
One of the weaknesses that is common to every computer-based 
decision-support system is a lack of "common sense" and a relative 
inability to consider important personal, social, family, and employment 
factors of the particular patient. This deficiency is well illustrated by 
an anecdote in the conclusion of Shortliffe's article. (Ref. 1) 

We believe it is critical that the physician retain the ultimate 
responsibility for identifying the correct diagnosis or diagnoses in any 
given patient. Using DXplain should be considered similar to consulting a 
medical textbook or journal article, DXplain should be used only as an 
adjunct, an information base, and a well-specified medical knowledge 
resource; DXplain cannot be a replacement for the clinician's knowledge and 
experience . 

CONCLUSIONS 

The potential contributions of computer-based decision-support systems 
are based on several factors: (1) the increasing complexity and scope of 
the medical knowledge base, (2) the increasing fragmentation and 
specialization of medical practice, (3) the increasing availability and 
af fordability of powerful computer technology, and (4) the increasing 
willingness among physicians to utilize computer technology in all phases 
of patient care activity. 

DXplain is an evolving computer system that uses an extensive 
knowledge base and relatively simple computational models to provide 
significant diagnostic problem-solving assistance to the practicing 
physician. The program allows the computer-naive user to enter a set of 
patient signs and symptoms and then generates a list of hypotheses that 
deserve consideration. The system also comments on the diagnostic relevance 
of each sign and symptom and suggests specific additional data elements 
that might clarify the differential diagnosis currently favored by the 
computer model. The advantages of a dynamic, interactive,' evolving 
reference tool, such as DXplain, over static, passive textbooks and journal 
articles are exciting. 

DXplain is unique in being a decision-support system that is easily 
and inexpensively available to a large number of physicians through 
nationwide medical information networks. A key element in the distribution 
of DXplain is the planned collaboration with its physician-users whose 
comments, criticisms, and suggestions will play an important role in 
modifying and enhancing the knowledge base and the algorithms used in the 
computer-generated interpretations . 

The most important evaluation of DXplain will be made by the 
physicians themselves as they attempt to use the system in their daily 
practice. As DXplain is used in a variety of practices with a variety of 
information needs, we will learn whether this system is capable of 
providing clinicians with useful information in a timely manner. The 
measure of success is not whether the ultimate diagnosis is listed first or 
fourth in the DXplain list of plausible diagnoses, but whether the 
physician-users perceive that the use of the system has added to their 
understanding of the patient's problems. Through careful analysis of user 
interactions, comments, level of satisfaction, and system performance, we 
expect to learn much about DXplain 's ability to provide assistance in 
making clinical decisions. Based on our experience during the testing 
phase, we are optimistic that DXplain will be a useful educational resource 
and an effective assistant to the physician in daily practice. 
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ABSTRACT: In the first part of a two-part article, the author discussed 
the problem faced by physicians and other health care workers trying to 
keep up with the explosion of new medical information. In this, the second 
part, he discusses in detail the contribution that computers are making to 
the management and dissemination of medical information. Medical knowledge 
stored in computers may be used in four ways, but the author indicates that 
these four ways are interrelated and tend to overlap. One application of 
computer medical knowledge is hypertext. In this system, the reader is not 
limited to reviewing text and illustrations in a particular order, but can 
jump from tidbit to tidbit in any order which suits his need. A drawback of 
such systems is that the freedom of movement within a body of medical 
knowledge may result in the reader becoming lost or perhaps not 
appreciating the relationships between concepts which would be presented 
sequentially in a bound volume. A second, and very important, application 
of computer technology to medical knowledge is simply the maintenance of 
large knowledge files. Though such files might be called "databases", they 
are in fact knowledge bases and are more akin to reference books than to 
databases. A third contribution of computer technology to medical knowledge 
is specialized teaching aids. These systems are designed not for the 
professional seeking to add to his knowledge or perhaps look up a fact of 
which he is not certain, but rather for people to learn new things. The 
computer provides a presentation which may be individually tailored to the 
needs and abilities of the learner. The last category of computer 
applications to medical knowledge are the decision aids. These systems, 
somewhat akin to expert systems, help the physician by presenting the 
step-by-step sequences of decisions which must be made in the proper 
diagnosis of some symptoms or in the management of continuing treatment of 
patients. (Consumer Summary produced by Reliance Medical Information, Inc.) 

TEXT: 

In the first article I discussed medical knowledge and 
decision-making, the sources that doctors currently use to find this 
knowledge, and some of the drawbacks of these sources. I also described 
computer-based methods for searching bibliographic files. In this paper I 
concentrate on novel computer systems in which medical knowledge is not 
only stored and manipulated as text meaningful to human beings but is also 
encoded as symbols meaningful to computers. 

There are four reasons to encode the knowledge held in a computer 
system: 

* To allow a computer user to move freely between passages of text, 
illustrations, or even sound recordings in a non-linear "hypertext" 
document . 

* To help a computer user find specific details in a large mass of 
medical facts stored in a "knowledge base", the electronic equivalent of a 
reference book. 

* To allow the computer to teach the user, by itself finding relevant 
details in its knowledge base that the user does not know (a teaching aid) . 

* To allow the computer to help the user solve clinical problems, by 
itself finding details in the knowledge base that are relevant to the 
patient {a medical decision aid) . 

The key properties of hypertext, knowledge bases, and teaching and 
decision aids are listed in the table. These four kinds of system tend to 
overlap: in general, the more helpful a system is in decision-making about 
individual patients, the narrower its coverage of medicine (fig 1). For 
simplicity, I shall look at each kind in isolation. 

Hypertext and hypermedia 



When we use a textbook, we do not usually start at the beginning and 
read on the end. Instead, we turn to figures or references on other pages, 
refer back to explanations in preceding chapters, and use glossaries and 
appendices to enhance our understanding. When text is stored on computer in 
the conventional way we are denied this instant access that the paper 
version gives us. However, these links can be recovered, and even enhanced, 
by the use of hypertext techniques [1] where links or short-cuts are 
provided between parts of the text. For example, a "definition" link may 
connect a word in the text with its glossary entry, while "see also" links 
draw attention to related sections of text. Frequently these links are 
shown as "buttons" or bold text on the computer screen, and users navigate 
through the system by selecting these items with a pointing device (mouse) . 
Development of these systems can be difficult, especially if there is more 
than one author, since one has to keep track of multiple passages of text 
and their links as the work goes through successivie revisions. Hypertext 
systems are now widely available outside medicine, guiding the uninitiated 
through software packages and tourists through airports. The Gosta's book 
project [2] is a medical hypertext system with 3000 links to help primary 
care physicians browse through the clinical and laboratory features of 
diseases . 

Hypertext alone is no substitute for books in which diagrams or 
photographs are important. Thus, developers are now incorporating 
high-resolution computer graphics, sound, and video pictures to produce 
"hypermedia" systems. These are most widely used in teaching, and I shall 
discuss them in the section on teaching aids. 

The similarity of one "page" to another in computer systems, the lack 
of a real beginning and end, and the fact that some hypertext systems offer 
ten or more links per page can lead to users getting "lost in hyperspace". 
To present a coherent overview and to avoid disorientation, some hypertext 
systems include a "guided tour" or maintain a list of all pages the user 
has viewed. Other systems provide an "overview" showing the structure of 
the whole document. [3] 

Medical knowledge bases 

Databases, the computer equivalent of card indexes, provide an 
efficient means for storing and retrieving large amounts of well structured 
data. If the store contains medical information such as drug dosages or 
disease incidences rather than patient data, the term "medical knowledge 

base " is preferred to database. There is a risk of overlap in terminology 
here, since disease incidences are calculated from a patient database, 
but few systems hold both kinds of information. Most knowledge bases 
include a search program to enable clinicians with modest computer 
experience to identify valid search items from lists and to combine them 
logically to identify relevant material. Medical knowledge bases can be 
stored on a central computer, with remote access by telephone or network 
links, or can be duplicated on floppy disk or compact disk read-only memory 
(CD-ROM) and installed on personal computers. 
[TABULAR DATA OMITTED] 

The difficulties of establishing and maintaining medical knowledge 
bases are identiacal to those associated with paper-based knowledge 
collections, discussed earlier. They include the difficulty of ensuring 
that the details contained are current and correct, that the coverage is 
adequate for clinical use, and that the index uses clinically relevant 
terms. However, two problems that are specific to electronic media are the 
use of symbols rather than a subtle natural language and the need to 
express knowledge in a form that can be read by any program running on any 
computer. 

The London Dysmorphology Database is an example of a medical 
knowledge base distributed on floppy disk. [4] This details the mode of 
inheritance and clinical signs of 2000 congenital malformation syndromes. 
An editorial panel regularly reviews 1000 journals and issues annual 
updates. If congenital malformation interest you, it is difficult to see 
how this system can be bettered. 

Floppy disks are not the ideal vehicle for knowledge bases because 
they are fragile and have limited capacity. A reasonable limit is 6, which 
will contain roughly 9 million characters equivalent to 1500 text pages. 
Instead, CD-ROMs are now widely used. These contain up to 660 million 
characters (the equivalent of 30 complete copies of the Oxford Textbook of 
Medicine), cannot be altered, and are physically more robust than floppy 



disks. One of the knowledge bases available on CD-ROM (medical librarians 
usually keep a catalogue) is OncoDisk: this includes an oncology textbook, 
the Physicians' Data Query, and a catalogue of cancer therapy protocols 
known to the US National Cancer Institute, with details of patient 
selection, monitoring, and treatment, and even the names of physicians 
using the protocol. Some physicians already carry a selection of such 
knowledge bases with a portable CD-ROM reader and a notebook computer, to 
help them answer questions about patient management. 

To keep a knowledge base current, updates must be sent out — a major 
enterprise if more than three or four a year are needed or the numbe of 
users is large. One solution is to allow access to a copy held and updated 
at a central point. An example is the Edinburgh Poisons Information System, 
which holds chemical and clinical data on thousands of poisons. (5] Access 
is gained through a Prestel terminal connected to a telephone line, and the 
system currently handles over 40 000 inquiries a year. Disadvantages of 
such dial-up knowledge bases are the need for special, sometimes arcane, 
search languages, and the slowness and cost of telephone links. 

To allow wider access to centrally held knowledge bases, there are 
proposals by the European Community and others for "open systems 
interconnection". Establishment of an electronic "global village" will 
require standardisation of the codes used for clinical terms: but linkage 
of heterogeneous medical nomenclatures such as the ICD-10, SNOMED, and Read 
clinical coding systems, even through the "Unified Medical Language 
System", [6] has yet to be achieved. The physical linking of computer 
systems is easier, with fast cable, fibreoptic, or cellular phone links 
(which allow portable computers to communicate with their base [7]), 

Teaching aids 

Teaching aids likewise include a medical knowledge base and programs 
to frame and execute searches. However, they are designed to enhance 
knowledge and understanding by presenting facts, explanations, and examples 
at the user's own level, thus superseding the old "teaching machines" that 
subjected all students to the same material. The knowledge base might 
consist of facts about a disease, or differential equations linking 
physiological variables, together with a search program. These are coupled 
with a program that elicits and records the facts a student does not know 
by asking suitable questions. The search program then assembles the unknown 
facts, together with suitable text, to display remedial material, before 
retesting the student. More elaborate systems provide a glossary of terms 
and offer static or video images and feedback through voice synthesisers; 
many use hypertext techniques. 

A slightly different approach is to use high-fidelity patient 
simulations, as in the "Mac" series. [8] 'MacMan', for example, consists of 
linked mathematical models describing the circulatory, respiratory, and 
fluid-balance systems with programs that allow the student to change key 
parameters and assess their effects on the simulated person; the 
mathematics are hidden behind an attractive graphic presentation which 
helps students to link their knowledge of anatomy and physiology and to 
learn about dynamic relations between body systems. [9] Students seem to 
like these commercial teaching programs. [10] 

One reason why the early teaching machines were badly received is 
that isolated or gratuitously offered facts are not well remembered: 
students learn best when they are confronted with a problem, [11] 
especially if they are emotionally involved in its solution. Some of the 
technologies outlined earlier make it easier to simulate cases in realistic 
fashion. For example, it is now possible to include short sections of video 
showing a "patient" in a hospital setting with authentic background noises 
punctuated by a nurse requesting advice, and then give the user guidance 
according to his or her responses to an on-screen question. One annoying 
aspect of these systems is that a typing error or the wrong vocabulary can 
be misconstrued as ignorance. Research is now focusing on more 
"intelligent" tutoring systems. 

The big drawback of all these teaching systems is the vast amount of 
work needed to assemble even a brief lesson, since there must be enough 
material to allow for all possible responses to each question. If the 
medium is interactive video, this means dozens, sometimes hundreds, of 
video clips, all of which must be filmed in the same location with the same 
actors. Consortia of medical schools and others are forming to build such 
programs, but new dilemmas have arisen such as the need for standards to 



allow interchange of material, and the question of who owns copyright. 
Medical decision aids 

A medical decision aid has a knowledge base and search program 
similar to that of a teaching system, but contains a program that builds a 
patient model rather than a student model. At its simplest, this program 
may ask the user to type in responses to questions about a patient such as 
age, height, weight, and lung function results, and then formulates a 
suitable search of its knowledge base . In this example, the knowledge 

base contains formulae to predict normal lung function test results and 
information about what abnormal results mean. [12] Instead of symbols 
representing medical details or strategy articulated by expert clinicians, 
the knowledge base of some systems contains a mathematical distillation 
of a series of cases with known outcomes, such as the prior probabilities 
of diseases and the conditional probabilities of certain findings in 
patients with those diseases , [13] The programs that build the patient 
model and search the knowledge base comprise a "reasoner", which 
reconciles the clinical features of a patient logically or mathematically 
with the knowledge in the system, to produce inferences. Such inferences 
might include a list of possible diagnoses with their probabilities, [14] 
potential adverse drug reactions, [15] or the therapy recommended by an 
oncology protocol. [13] 

The first computerised medical decision aid was built in the late 
1950s, [16] and thousands have been made since then. They are best 
classified by their clinical role. One type interprets patient 
data — sounding an alarm, reporting on an ECG, [17] or highlighting abnormal 
laboratory test results. [12] The second kind requests the user to enter 
patient data, and proposes a differential diagnosis [18] or management 
plan. Decision aids of this sort are the commonest today, but they resemble 
a Greek oracle in their reluctance to allow their knowledge to be used for 
other purposes. [19] The third type allows a more flexible interaction, 
with more sophisticated search and model-building programs, so that user 
and system can negotiate jointly for solutions. [20,21] 

Most medical decision aids use one of three reasoning methods — Bayes' 
theorem, decision theory, or symbolic reasoning or "artificial 
intelligence" techniques. Bayes' theorem tells us how to use probabilities 
derived from a database of past cases with a known outcome to calculate the 
probability of the outcome in an individual case. [14] Decision theory is 
an extension to Bayes' theorem that lets us add weightings representing the 
value of each possible outcome to a decision tree, to calculate the 
"expected utility". [22] Symbolic reasoning methods include the use of "if 
, . . then" rules and other complex computer models [23] to link clinical 
findings with their causes and management. All these reasoning methods have 
been refined since their introduction and are now being combined — for 
example, in directed acyclic graphs, [24] one of which [25] is shown in 
fig. 2. Other reasoning methods include obscure computational models 
inspired by the structure of brain. [26] These "neural nets" rely on a 
process called machine-learning, and their evaluation is far from easy. 
[27] 

The patient simulators used by students can also be helpful to 
clinicians. For example, a program that predicts blood glucose profile over 
24 hours with a specific insulin regimen [28] may allow a doctor to explore 
alternative managements for a patient in a series of computerised 
experiments that hold no risk for the patient. 

Few of the developers of medical decision aids see them as anything 
other than animated reference books, complementing rather than replacing 
human knowledge. Not so Weed with his "Problem-Knowledge Coupler". [29] A 
coupler is a computer program designed to provide clinicians with all the 
medical details they need to tackle a specific clinical problem, without 
committing medical facts to memory. Couplers are available for some 15 
common conditions at present: if widely adopted, such systems might have a 
revolutionary impact on medical education, but would leave their authors in 
a position of unique responsibility. 

Medical decision aids have already proved themselves clinically, with 
the Leeds Abdominal Pain System halving perforated-appendix rates in a 
multi-centre study, [30] Pozen's chest pain system leading to a reduction 
in inappropriate admissions to a coronary care unit, [31] and ONCOCIN (a 
symbolic reasoner for assisting in protocol-driven care) improving the 
completeness of data collection in cancer patients. [32] Systems are being 



built to address a clinical need, [33] but may raise complex medicolegal 
questions for their users and those who supply them. [34] However, there 
are also more immediate dilemmas for users and developers. 
Limited scope of decision aids 

No medical decision aid contains as much knowledge as a medical 
textbooks, and the intellectual and practical challenges of building and 
maintaining large knowledge bases to support doctors remain daunting. 
Although it is possible to express lower-level medical details in knowledge 
bases, making higher-level knowledge explicit is a complex "knowledge 
engineering" task that requires understanding about how to acquire, 
represent, and reason with knowledge. [23] There are similar problems with 
building large databases of patient cases and extracting summary measures 
from them to reason about new patients, who may not be drawn from a similar 
population. [35] 

Some decision aids, however, have achieved good coverage of medicine. 
The INTERNIST system performed acceptably in diagnosis of a range of 
clinico-pathological cases from the New England Journal of Medicine. [18] 
It is now commercially available as 'QMR' on personal computers, covering 
over 600 diseases and 4300 patient findings. DXPlain is another 
broad-ranging system that was widely available through a computer network. 
[36] In Britain, the 'Oxford System of Medicine' is planned as a 
comprehensive flexible information system for general practitioners, and a 
prototype containing a fraction of the required knowledge has demonstrated 
that the project is technically feasible. [21] One alternative to building 
a single large knowledge base and then adding a reasoner is to 
integrate several different decision aids, perhaps even using different 
reasoning methods, into one flexible resource, such as the EXPLORER system 
built at Harvard. [3] 

Difficulties of communicating with computers 

Many people become frustrated when trying to persuade computers to do 
what is wanted. When a doctor feeds patient data into a decision aid, a 
good system will not only communicate its advice back but also convince the 
user that the advice is well based. With the 'QMR' system it took an 
experienced user between one and four hours to enter enough patient data to 
generate a useful differential diagnosis. [37] 

Most decision aids communicate only by text. ' IntelliPath ' , however, 
includes a laser videodisk storing 5000 photomicrographs. [38] These are 
used to augment text definitions of histological features and to show which 
features best distinguish between possible diagnoses. Some 300 copies of 
the system, which is supported by the- American Society of Clinical 
Pathology, have been distributed. Other methods that may assist in rapid 
and accurate data entry include graphic "user interfaces" (fig 3), 
handwriting and speech recognition, [39] and direct interviewing of the 
patient by computer. [14] The ideal, however, is to integrate decision aids 
with an electronic medical record, so that all relevant data are then 
available, as with the HELP system. [40] 

How do decision aids explain their advice to the physician? One 
system allocates "weights of evidence" to each clinical finding; [14] 
others use their symbolic knowledge about anatomy, pathophysiology, or 
mechanisms of drug action to generate textual explanations of their 
reasoning; [23] yet others are able to explain not only how they arrived at 
their advice, but also how it would have differed had certain patient 
features been absent, or why an alternative explanation is implausible. 
However, as with intelligent tutoring systems, it is often difficult for 
the system to understand what kind of explanation is required. 

Need for evaluation 

A major concern is that computers, like doctors, are often unable to 
admit their ignorance. They also have enormous placebo power, exacerbated 
by the term "expert system", which may cause inexperienced doctors to 
accept their advice without question. Before any system is used to support 
patient care, we must be confident that it is giving sensible advice. The 
content of a decision aid's knowledge base, the rigour of its reasoning, 
and its performance on suitable test cases must all be examined. [41] Even 
if the decision aid's performance is promising, it may still lack 
beneficial impact if it is difficult to use or gives implausible advice: 
[42] as with drugs, controlled clinical trials are the only way to assess 
their impact on doctors and patients. [41] 

An ideal knowledge system 



What every doctor needs is a portable, accurate, and adaptable source 
of medical knowledge. At present, nothing of the kind exists; but it is not 
too soon to think about the features required of such a system. What 
foil ows is based partly on the surveys of doctors discussed in the first 
paper. [43-45] 

The system must be comprehensive 

The system should cover the majority of diseases, therapies, and 
investigations as well as basic medical sciences, and should integrate 
text, graphics, moving images, and sound. 

Knowledge contained in the system must be current, 

accurate, and verifiable 

The certainty of the details stored should be stated, together with 
their sources to allow easy verification. The system should maintain its 
own record of the age of its knowledge, and updates should be issued 
regularly; all information should be safeguarded against unauthorised 
modification. To avoid ambiguity and to ease maintenance, a consistent 
vocabulary should be used, possibly even a logical language. [21] 

Knowledge in the system must be easily accessible, 

where doctors see patients 

The system should be portable and unobtrusive — preferably fitting 
into a white-coat pocket — and also cheap, indestructible, and workable 
without special equipment. To assist in clinical decision making, it must 
be fast and easy to use. Instead of a typewriter keyboard which is bulky 
and slow, the system should incorporate hand-writing recognition (like the 
new generation of pen-input computers) . [46] The indexing system should 
include abbreviations and synonyms used in clinical practice; cross 
references to MeSH would facilitate literature searches. To help users 
absorb new material, updates should include digestible text overviews, 
analogous to the reviews and editorials in the medical weeklies. 

The system should be adaptable 

Since clinicians will be using the system throughout their working 
day, it should accommodate electronic notes and be adaptable to their own 
preferred terms or abbreviations. This could be achieved if each doctor 
carried a personal electronic badge which broadcast their identity and 
other data to the system in their hands. [7] The system should also be able 
to adapt itself to specific clinical problems, tailoring its reports and 
advice according to patient features. To do this, it could be linked by 
cellular radio networks to clinical data systems. This would also help it 
to predict, from information on who was using it and where, what knowledge 
the physician might need next--"ant icipatory computing". [47] 

Conclusions 

There are few systems that meet even one of the above criteria, but 
advances in information technology will make the ideal knowledge system 
technically feasible. Furthermore, developments in medicine itself, and in 
the administrative framework surrounding it, suggest that the intellectual* 
and practical challenges will be met. 

One development is that medical societies are taking increasing 
responsibility for their members' knowledge as well as their professional 
competence. [48] Thus, specialty societies are defining management 
protocols, minimum data sets, and the content of coding systems. However, 
establishing and maintaining a fund of medical knowledge is a major 
undertaking, and few specialty bodies will have the resourced to do it. 

Another development is the establishment of medical informatics as 
the discipline concerned specifically with the acquisition, storage, and 
communication of medical knowledge and data. [49,50] Training courses, 
academic departments, professional journals, and societies now exist, [51] 
together with a definitive textbook. [52] The big issues in medical 
informatics include how to represent and store medical knowledge 
electronically and how to help doctors extract knowledge from these stores 
and apply it to their patients. Collaboration with specialists in medical 
education, computer science, and medical cognitive science [53] will be 
necessary to resolve some of these questions. A further challenge is to 
find ways of evaluating and comparing the sources of knowledge, and of 
assessing their impact on medical decisions (and on patients) . 

If the medical profession could harness the medical, technical, and 
managerial skills and produce a system such as that described above, the 
quality of medical care could take a quantum leap forward. Some clinicians 
may argue that reliance on such support will degrade their skills. This was 



the cry when Laennec introduced the stethoscope. 

These two papers were written partly at the National Heart & Lung^ 
Institute, London, and partly while I held an MRC travelling fellowship at 
the Section on Medical Informatics, Stanford University. I gratefully 
acknowledge the assistance of colleagues at both sites, others who have 
contributed their time, and the staff of the Lister Hill National Center 
for Biomedical Communications, US National Library of Medicine. 
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TEXT: 

SAN FRANCISCO-- (BW HealthWire ) --March 17, 1997— UC San Francisco AIDS 
specialists announced today (March 17) that anyone with access to a 
computer and to the "world wide web" can now obtain the most comprehensive, 
credible and trustworthy HIV/AIDS information available online with just a 
click of the mouse. 

The top-rated UCSF AIDS Program at San Francisco General Hospital and 
the UCSF Center for AIDS Prevention Studies (CAPS), programs of the UCSF 
AIDS Research Institute, have created "HIV InSite" — a one-stop shop for 
reliable, peer-reviewed AIDS information and the only website in existence 
that contains research written, edited and maintained by frontline AIDS 
researchers from a health sciences institution. 

The address for the site is http : //hivinsite . ucsf , edu . 

"Reliable, timely information is essential to developing effective 
responses to the HIV epidemic, " said Paul Volberding, MD, UCSF professor of 
medicine and director of the UCSF AIDS Program at SFGH. "The internet 
offers the potential to arm decision makers throughout the world with 
accurate up-to-date information, fostering the development of sound 
policies, enhancing prevention program development, and providing people 
living with HIV with information about treatments that may extend their 
lives . " 

Unfortunately, AIDS information reported in the popular media is 
often incomplete or inaccurate, and those most in need of information may 
sometimes become overwhelmed or lost in cyberspace, according to Thomas 
Coates, PhD, director of CAPS and the UCSF AIDS Research Institute. 

"Information found at HIV Insite is reviewed and presented by the top 
thinkers in the AIDS field," said Coates. "No other institution can offer 
such depth in this area." 

"With the launch of HIV InSite, patients, physicians, researchers, 
policymakers, health service providers, community organizers and 
journalists can be ensured thaf "they are accessing the most reliable and 
current online resource for AIDS knowledge, " Coates added. 

Surfers of internet and users of "HIV InSite" will benefit from the 
breadth of in-depth information on the site — from treatment, clinical 
drug trials, epidemiology and basic research to social and policy issues, 
prevention programs, population subgroups, and ethics. A distinct advantage 
of HIV InSite is the ability of the user to "see the big picture" of the 
AIDS topic he or she is researching, thereby adding value to the search, 
said Nicole Mandel, project manager of the web site . 

For example, a reporter in New Mexico who might be writing a story 
about HIV infection among teenage runaways may dialup the internet to find 
out prevalence statistics. Accessing HIV InSite, the reporter will not only 
find out HIV rates among this group, which can be broken down by regions or 
states, but will find facts on homelessness and AIDS, a research study on 
HIV prevention for high-risk youth, and lists of local and national 
advocacy groups working with young people. 

Another scenario might include an outreach worker at a 
community-based organization in North Dakota who is working with prisoners 
for the first time. Visiting HIV InSite, he or she will find a list of 
prison-based HIV prevention projects, an evaluation of a post-incarceration 
followup programs at San Quentin, and a review of prison condom 
distribution programs. 

"The comprehensive nature of HIV Insite adds value to the 
researchers* information search," Mandel said. 

Pages or subcategories of HIV Insite include: 
Medical 



This area provides comprehensive, practical and state-of-the-art 
information to meet the needs of health care providers, clinical 
researchers, health care policymakers and those living with HIV disease . 
Highlights include a "trials search" — a user-friendly database of all HIV 
clinical trials in the U.S. in which users may search by medical 
condition , drug or treatment name, or location to find locations of 
appropriate clinical trials. Another feature is the AIDS Knowledge Base 
— considered by many researchers to be the "bible" of HIV disease — a 
1600-page online textbook that is edited by AIDS specialists from UCSF and 
SFGH. In addition, users may explore case studies, treatment guidelines, 
and clinical fact sheets. 
Prevention 

The goal of this section is to help service providers, researchers, 
educators and others build stronger programs and studies in the effort to 
prevent HIV infections. HIV InSite offers detailed resources to answer some 
of the most pressing needs in prevention, including pages on epidemiology, 
prevention basics, frequently asked questions and their answers, resources 
and links to basic information about HIV/AIDS and safer sex, and HIV 
prevention interventions that have been evaluated for effectiveness and 
have been published in scientific journals and government reports. 
Social Issues 

This section is a searchable, comprehensive index of HIV social and 
ethical issues, resources and analysis. The site contains policy resource 
materials, such as statements, guidelines, reports and analysis, and new 
developments on topics ranging from adolescents to workplace issues. The 
section catalogues and organizes policy information and resources from 
numerous sources, including federal, state and local legislatures and 
agencies, the courts, professional associations, health policy research 
organizations, AIDS community-based organizations and advocacy groups. 
U.S. Map 

A map of the United States includes state-by-state statistics of the 
epidemic, as well as links to AIDS agencies and resources available in 
individual states. The AIDS epidemic in the U.S. varies by metropolitan 
area and state, and accounts for a total of 7% of all cases of HIV 
infection in the world. 

HIV InSite will expand this geographic guide to HIV to include other 
areas around the globe, but in the meantime has links to other HIV InSite 
pages or other websites that have international data. HIV-infected 
individuals will also be able to locate clinical drug trials in their 
geographic area by keying in their medical information either through the 
U.S. map or through the clinical section of HIV InSite. 
Key Topics 

This section includes comprehensive information on key HIV and AIDS 
topic areas. Users may look here for news, articles, opinion pieces, 
documents, abstracts, bibliographies, contacts and other information. Among 
topics included in the exhaustive list are: advances in treatment, African 
TVmericans, condoms, discrimination, health care workers, international, 
lesbians, Native American, needle exchange, Ryan White Care Act, schools, 
sex workers, tuberculosis, vaccines and women. 

The UCSF AIDS Program at San Francisco General Hospital, founded in 
1983, is one of the oldest and largest programs of its kind in the world. 
With an internationally-renowned faculty and extensive clinical and 
research activities, the AIDS program is on the frontline of advances in 
clinical care for people with HIV. The program received the number one 
ranking from U.S. News and World Report in 1996 for the sixth year in a 
row . 

CAPS, established in 1986, is the largest consolidated research 
effort in the country focusing on the social and behavioral aspects of AIDS 
prevention and early intervention. Research activities are carried out 
locally, nationally and internationally, 

CAPS also conducts ethical studies and policy analyses of 
AIDS-related issues, provides a program of technology transfer and exchange 
with community-based organizations, and trains U.S. and international 
scientists in AIDS prevention. CAPS is made possible through grant funding 
from the Office of AIDS and the National Institute of Mental Health. 

HIV InSite is made possible by a grant from the Henry J. Kaiser 
Family Foundation, an independent health care philanthropy based in Menlo 
Park, California. The Foundation is not associated with Raiser Permanente 



or Kaiser Industries. The Foundation supports research in four main areas: 
health policy, reproductive health, HIV policy, and health and development 
in South Africa. The Foundation also maintains a special interest in health 
policy and innovation in its home state of California. 

Support for HIV InSite was also provided in the form of unrestricted 
educational grants from Bristol-Myers Squibb Company, Merck and Co., and 
Sun Microsystems. -0- 

Note To Editors: To receive a press kit about "HIV InSite" and/or to 
interview website authors and editors, media should call UCSF News Services 
at 415/476-2557. 

CONTACT: UCSF News Service 

Rebecca Higbee, 415/476-2557 
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